Communication system, transmission apparatus, receiving apparatus, communication method, and program

ABSTRACT

The present invention relates to a communication system capable of performing efficient data transmission in HDMI(R) and the like, a transmission apparatus, a receiving apparatus, a communication method, and a program. 
     An HDMI(R) source  53  determines whether or not an HDMI(R) sink  61  can receive a sub-signal on the basis of VSDB of E-EDID. When the HDMI(R) sink  61  can receive the sub-signal, the HDMI(R) source  53  adds a sub-signal to pixel data of a main image composed of pixel data whose number of bits is smaller than that of transmission pixel data transmitted by a transmitter  72 , thereby constructing transmission pixel data. This data is transmitted by the transmitter  72  through TMDS channels # 0  to # 2 . Furthermore, the HDMI(R) source  53  transmits a general control packet containing sub-signal information indicating whether or not the sub-signal is contained in the transmission pixel data in the control period of a vertical blanking period. The present invention can be applied to, for example, HDMI(R).

TECHNICAL FIELD

The present invention relates to a communication system, a transmissionapparatus, a receiving apparatus, a communication method, and a program.More particularly, the present invention relates to a communicationsystem capable of performing efficient data transmission using, forexample, a communication interface, such as HDMI (High DefinitionMultimedia Interface) (R), which is capable of transmitting pixel dataof an uncompressed image in one direction at high speed, to atransmission apparatus and a receiving apparatus, to a communicationmethod, and to a program.

BACKGROUND ART

In recent years, for example, as a communication interface fortransmitting a digital television signal, that is, pixel data of anuncompressed (baseband) image (moving image) and audio data accompaniedwith the image, at high speed from a DVD (Digital Versatile Disc)recorder, a set-top box, or another AV source (source) to a televisionreceiver, a projector, or another display, HDMI(R) has been becomingpopular.

Regarding HDMI(R), a TMDS (Transition Minimized Differential Signaling)channel for transmitting pixel data and audio data at high speed from anHDMI(R) source to an HDMI(R) sink in one direction, a CEC (ConsumerElectronics Control) line for performing bidirectional communicationbetween the HDMI(R) source and the HDMI(R) sink, and the like have beendefined in the specification of HDMI (the current up-to-datespecification: “High-Definition Multimedia Interface SpecificationVersion 1.2a”, Dec. 14, 2005).

Furthermore, in HDMI(R), HDCP (High-Bandwidth Digital ContentProtection) can be implemented to prevent copying of content.

In addition, for the HDMI(R), a method for not transmitting anunnecessary signal in a vertical blanking period and in a horizontalblanking period has been proposed (refer to, for example, PatentDocument 1).

[Patent Document 1] Japanese Unexamined Patent Application PublicationNo. 2005-102161

DISCLOSURE OF INVENTION Problems to be Solved by the Invention

In the current HDMI(R), for example, an image, RGB (Red, Green, Blue)color data of which is each composed of 8-bit pixel data (hereinafteralso referred to as a 24-bit (=8 bits× . . . 3) image as appropriate),can be transmitted. In recent years, there has been an increasing demandfor transmitting an image having a higher resolution, that is, an imagein which each of RGB data is composed of pixel data of a large number ofbits such as 10 bits or 12 bits, which is larger than 8 bits(hereinafter also referred to as a high resolution image).

Accordingly, in HDMI(R), a method of transmitting a high resolutionimage has been studied.

However, although there has been an increasing demand for transmitting ahigh resolution image, 24-bit images are still often handled.

Therefore, in the case that HDMI(R) is extended so as to be capable oftransmitting a high resolution image, when a 24-bit image is to betransmitted, unnecessary data is transmitted in an amount correspondingto the difference between the number of bits of the pixel data of thehigh resolution image and the 24 bits, which is the number of bits ofthe 24-bit image, per pixel. Thus, inefficient data transmission isperformed.

The present invention has been made in view of such circumstances. It isan object of the present invention to be capable of performing efficientdata transmission using, for example, a communication interface such asHDMI(R), which is capable of transmitting pixel data of an uncompressedimage at high speed in one direction.

Means for Solving the Problems

A first aspect of the present invention provides a transmissionapparatus including a transmission apparatus for, after capabilityinformation indicating the capability of a receiving apparatus isreceived, transmitting pixel data of an uncompressed image for onescreen in one direction to the receiving apparatus by using adifferential signal through a plurality of channels for transmittingdata of a fixed number of bits per clock of a pixel clock in a validimage period that is a period in which a horizontal blanking period anda vertical blanking period are excluded from a period from one verticalsynchronization signal to the next vertical synchronization signal; andthe receiving apparatus for receiving the pixel data transmitted using adifferential signal through the plurality of channels from thetransmission apparatus after the capability information is transmitted,wherein the transmission apparatus includes transmission means fortransmitting pixel data to which a number of bits, which is greater thanthe fixed number of bits, is assigned, in one direction to the receivingapparatus by using a differential signal through the plurality ofchannels by adjusting the frequency of the pixel clock; sub-signalreception capability/incapability determination means for determiningwhether or not the receiving apparatus can receive a sub-signal on thebasis of the capability information; sub-signal addition means foradding the sub-signal to pixel data of a main image composed of pixeldata whose number of bits is smaller than that of transmission pixeldata that is pixel data to be transmitted by the transmission means whenthe receiving apparatus can receive the sub-signal, thereby constructingthe transmission pixel data; and information transmission control meansfor allowing sub-signal information indicating whether or not thesub-signal is contained in the transmission pixel data transmitted inthe valid image period immediately after the vertical blanking period,to be transmitted in the vertical blanking period, and wherein thereceiving apparatus includes receiving means for receiving thetransmission pixel data transmitted using a differential signal throughthe plurality of channels; sub-signal presence/absence determinationmeans for determining whether or not the sub-signal is contained in thetransmission pixel data transmitted in a valid image period immediatelyafter the vertical blanking period on the basis of the sub-signalinformation transmitted in the vertical blanking period; and separationmeans for separating the sub-signal from the transmission pixel datawhen the sub-signal is contained in the transmission pixel data.

In the communication system of the above-described first aspect, in thetransmission apparatus, the transmission means adjusts the frequency ofthe pixel clock and thereby transmits pixel data to which a number ofbits greater than the fixed number of bits is assigned to the receivingapparatus in one direction by using a differential signal through theplurality of channels. It is determined whether or not the receivingapparatus can receive a sub-signal on the basis of the capabilityinformation. When the receiving apparatus can receive the sub-signal,the transmission pixel data is constructed by adding the sub-signal tothe pixel data of the main image formed of pixel data having a number ofbits less than that of the transmission pixel data that is pixel data tobe transmitted by the transmission means. In the vertical blankingperiod, sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data that is transmitted in avalid image period immediately after the vertical blanking period istransmitted. On the other hand, in the receiving apparatus, thereceiving means receives the transmission pixel data transmitted using adifferential signal through the plurality of channels. Furthermore, onthe basis of the sub-signal information transmitted in the verticalblanking period, it is determined whether or not the sub-signal iscontained in the transmission pixel data transmitted in the valid imageperiod immediately after the vertical blanking period. When thesub-signal is contained in the transmission pixel data, the sub-signalis separated from the transmission pixel data.

A second aspect of the present invention provides a transmissionapparatus for, after capability information indicating the capability ofa receiving apparatus is received, transmitting pixel data of anuncompressed image for one screen in one direction to the receivingapparatus by using a differential signal through a plurality of channelsfor transmitting data of a fixed number of bits per clock of a pixelclock in a valid image period that is a period in which a horizontalblanking period and a vertical blanking period are excluded from aperiod from one vertical synchronization signal to the next verticalsynchronization signal, the transmission apparatus including:transmission means for transmitting pixel data to which a number ofbits, which is greater than the fixed number of bits, is assigned, inone direction to the receiving apparatus by using a differential signalthrough the plurality of channels by adjusting the frequency of thepixel clock; sub-signal reception capability/incapability determinationmeans for determining whether or not the receiving apparatus can receivea sub-signal on the basis of the capability information; sub-signaladdition means for adding the sub-signal to pixel data of a main imagecomposed of pixel data whose number of bits is smaller than that oftransmission pixel data that is pixel data to be transmitted by thetransmission means when the receiving apparatus can receive thesub-signal, thereby constructing the transmission pixel data; andinformation transmission control means for allowing sub-signalinformation indicating whether or not the sub-signal is contained in thetransmission pixel data transmitted in the valid image periodimmediately after the vertical blanking period, to be transmitted in thevertical blanking period.

A second aspect of the present invention provides a communication methodfor use with a transmission apparatus for transmitting pixel data of anuncompressed image for one screen in one direction to a receivingapparatus by using a differential signal through a plurality of channelsfor, after capability information indicating the capability of thereceiving apparatus is received, transmitting data of a fixed number ofbits per clock of a pixel clock in a valid image period that is a periodin which a horizontal blanking period and a vertical blanking period areexcluded from a period from one vertical synchronization signal to thenext vertical synchronization signal, or provides a program to beexecuted by a computer for controlling the transmission apparatus, thetransmission apparatus including transmission means for transmittingpixel data to which a number of bits, which is greater than the fixednumber of bits, is assigned, in one direction to the receiving apparatusby using a differential signal through the plurality of channels byadjusting the frequency of the pixel clock,

the communication method including the steps of: determining whether ornot the receiving apparatus can receive a sub-signal on the basis of thecapability information; adding the sub-signal to pixel data of a mainimage composed of pixel data whose number of bits is smaller than thatof transmission pixel data that is pixel data to be transmitted by thetransmission means when the receiving apparatus can receive thesub-signal, thereby constructing the transmission pixel data; andallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period.

In the above-described second aspect, on the basis of the capabilityinformation, it is determined whether or not the receiving apparatus canreceive a sub-signal. When the receiving apparatus can receive thesub-signal, the transmission pixel data is constructed by adding thesub-signal to the pixel data of the main image formed of pixel datahaving a number of bits less than that of the transmission pixel datathat is pixel data to be transmitted by the transmission means. In thevertical blanking period, sub-signal information indicating whether ornot the sub-signal is contained in the transmission pixel data that istransmitted in a valid image period immediately after the verticalblanking period is transmitted.

A third aspect of the present invention provides a receiving apparatusfor receiving pixel data transmitted using a differential signal througha plurality of channels from a transmission apparatus after capabilityinformation is transmitted to the transmission apparatus fortransmitting pixel data of an uncompressed image for one screen in onedirection to the receiving apparatus by using a differential signalthrough a plurality of channels for, after capability informationindicating the capability of the receiving apparatus is received,transmitting data of a fixed number of bits per clock of a pixel clockin a valid image period that is a period in which a horizontal blankingperiod and a vertical blanking period are excluded from a period fromone vertical synchronization signal to the next vertical synchronizationsignal, the transmission apparatus including transmission means fortransmitting pixel data to which a number of bits, which is greater thanthe fixed number of bits, is assigned, in one direction to the receivingapparatus by using a differential signal through the plurality ofchannels by adjusting the frequency of the pixel clock; sub-signalreception capability/incapability determination means for determiningwhether or not the receiving apparatus can receive a sub-signal on thebasis of the capability information; sub-signal addition means foradding the sub-signal to pixel data of a main image composed of pixeldata whose number of bits is smaller than that of transmission pixeldata that is pixel data to be transmitted by the transmission means whenthe receiving apparatus can receive the sub-signal, thereby constructingthe transmission pixel data; and information transmission control meansfor allowing sub-signal information indicating whether or not thesub-signal is contained in the transmission pixel data transmitted inthe valid image period immediately after the vertical blanking period,to be transmitted in the vertical blanking period, the receivingapparatus including: receiving means for receiving the transmissionpixel data transmitted using a differential signal through the pluralityof channels; sub-signal presence/absence determination means fordetermining whether or not the sub-signal is contained in thetransmission pixel data transmitted in a valid image period immediatelyafter the vertical blanking period on the basis of the sub-signalinformation transmitted in the vertical blanking period; and separationmeans for separating the sub-signal from the transmission pixel datawhen the sub-signal is contained in the transmission pixel data.

A third aspect of the present invention provides a communication methodfor use with a receiving apparatus for receiving pixel data transmittedusing a differential signal through a plurality of channels from atransmission apparatus after capability information is transmitted tothe transmission apparatus for, after capability information indicatingthe capability of the receiving apparatus is received, transmittingpixel data of an uncompressed image for one screen in one direction tothe receiving apparatus by using a differential signal through aplurality of channels for transmitting data of a fixed number of bitsper clock of a pixel clock in a valid image period that is a period inwhich a horizontal blanking period and a vertical blanking period areexcluded from a period from one vertical synchronization signal to thenext vertical synchronization signal, the transmission apparatusincluding transmission means for transmitting pixel data to which anumber of bits, which is greater than the fixed number of bits, isassigned, in one direction to the receiving apparatus by using adifferential signal through the plurality of channels by adjusting thefrequency of the pixel clock; sub-signal receptioncapability/incapability determination means for determining whether ornot the receiving apparatus can receive a sub-signal on the basis of thecapability information; sub-signal addition means for adding thesub-signal to pixel data of a main image composed of pixel data whosenumber of bits is smaller than that of transmission pixel data that ispixel data to be transmitted by the transmission means when thereceiving apparatus can receive the sub-signal, thereby constructing thetransmission pixel data; and information transmission control means forallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period, or provides a program to beexecuted by a computer for controlling the receiving apparatus, thereceiving apparatus including receiving means for receiving thetransmission pixel data transmitted using a differential signal throughthe plurality of channels, the communication method including the stepsof: determining whether or not the sub-signal is contained in thetransmission pixel data transmitted in a valid image period immediatelyafter the vertical blanking period on the basis of the sub-signalinformation transmitted in the vertical blanking period; and separatingthe sub-signal from the transmission pixel data when the sub-signal iscontained in the transmission pixel data.

In the above-described third aspect, on the basis of the sub-signalinformation transmitted in the vertical blanking period, it isdetermined whether or not the sub-signal is contained in thetransmission pixel data transmitted in the valid image periodimmediately after the vertical blanking period. When the sub-signal iscontained in the transmission pixel data, the sub-signal is separatedfrom the transmission pixel data.

ADVANTAGES

According to the first to third aspects of the present invention,efficient data transmission can be performed using, for example, acommunication interface such as HDMI(R), which is capable oftransmitting pixel data of an uncompressed image at high speed in onedirection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of the configuration of anembodiment of an AV system to which the present invention is applied.

FIG. 2 is a block diagram showing an example of the configuration of anHDMI(R) source 53 and an HDMI(R) sink 61.

FIG. 3 is a block diagram showing an example of the configuration of atransmitter 72 and a receiver 81.

FIG. 4 shows a transmission period of transmission through three TMDSchannels #0 to #2.

FIG. 5 is a timing chart showing the relationship among control bitsCTL0 and CTL1, a data island period, and a control period.

FIG. 6 is a timing chart showing the timing of transmission of the pixeldata of an image transmitted in a video data period of the currentHDMI(R).

FIG. 7 is a timing chart showing the timing of transmission of pixeldata when a 30-bit image is transmitted in the video data period ofHDMI(R).

FIG. 8 is a timing chart showing the timing of transmission of pixeldata when a 36-bit image is transmitted in the video data period ofHDMI(R).

FIG. 9 is a timing chart showing the timing of transmission of pixeldata when a 48-bit image is transmitted in the video data period ofHDMI(R).

FIG. 10 shows the format of VSDB in E-EDID.

FIG. 11 shows the format of a general control packet.

FIG. 12 shows the relationship between bits CD0, CD1, and CD2 of byte#SB1 of a subpacket, and an image transmitted in a video data period.

FIG. 13 is a flowchart illustrating the operation of an HDMI(R) source53 in compliance with deep color HDMI(R).

FIG. 14 is a flowchart illustrating the operation of an HDMI(R) sink 61in compliance with deep color HDMI(R).

FIG. 15 illustrates the transmission of pixel data of a main imagehaving a resolution lower than that of an image in a deep color modedetermined by the HDMI(R) source 53.

FIG. 16 illustrates an assignment method of assigning a sub-signal totransmission pixel data.

FIG. 17 shows the format of VSDB in E-EDID.

FIG. 18 shows the format of a general control packet.

FIG. 19 shows the relationship between bits SD0, SD1, and SD2 of byte#SB2 of a subpacket, and the number of bits of a sub-signal.

FIG. 20 is a block diagram showing an example of the configuration of asource signal processor 71.

FIG. 21 illustrates sub-signal-related information contained in asub-signal.

FIG. 22 is a flowchart illustrating the operation of the HDMI(R) source53 in compliance with extended HDMI(R).

FIG. 23 is a block diagram showing an example of the configuration of asink signal processor 82.

FIG. 24 is a flowchart illustrating the operation of the HDMI(R) sink 61in compliance with extended HDMI(R).

FIG. 25 is a block diagram showing an example of the configuration of anembodiment of a computer to which the present invention is applied.

FIG. 26 is a block diagram showing an example of a system configurationaccording to an embodiment of the present invention.

FIG. 27 is an illustration showing an example of the configuration of atransmission channel according to an embodiment of the presentinvention.

FIG. 28 is an illustration showing an example of bit structure accordingto an embodiment of the present invention.

FIG. 29 is an illustration showing an example of a data packing example(example 1) according to an embodiment of the present invention.

FIG. 30 is an illustration showing an example of a data packing example(example 2) according to an embodiment of the present invention.

FIG. 31 is an illustration showing an example of the data structure ofVSDB according to an embodiment of the present invention.

FIG. 32 is an illustration showing an example of display of a main imageand a sub-image according to an embodiment of the present invention.

FIG. 33 is an illustration showing an example of data packing (exampleof 8 bits per pixel) of the HDMI standard.

FIG. 34 is an illustration showing an example of data packing (exampleof 10 bits per pixel) of the HDMI standard.

FIG. 35 shows RGB 4:4:4 for 24-bit color depth.

FIG. 36 shows a signal mapping and a timing for transferring 24-bitYC_(B)C_(R) 4:2:2 data in HDMI(R).

FIG. 37 shows a signal mapping and a timing for transferring 24-bitYC_(B)C_(R) 4:4:4 data in HDMI(R).

FIG. 38 shows all “pixel coding” for all color depths.

FIG. 39 shows a group size, and a sequence of HSYNC and VSYNCtransmission within a group for a 24-bit mode.

FIG. 40 shows a group size, and a sequence of HSYNC and VSYNCtransmission within a group for a 30-bit mode.

FIG. 41 shows a group size, and a sequence of HSYNC and VSYNCtransmission within a group for the remainder of a 30-bit mode.

FIG. 42 shows a group size, and a sequence of HSYNC and VSYNCtransmission within a group for a 36-bit mode.

FIG. 43 shows a group size, and a sequence of HSYNC and VSYNCtransmission within a group for a 48-bit mode.

FIG. 44 shows color depth values (CD fields) of SB1.

FIG. 45 shows specific PP values regarding each packing phase shown in apacking phase table in an early period.

FIG. 46 shows YC_(B)C_(R) 4:2:2 of pixel-doubling.

FIG. 47 shows a video color component range.

FIG. 48 is a state machine diagram for a 24-bit mode.

FIG. 49 is a state machine diagram for a 30-bit mode.

FIG. 50 is a state machine diagram for a 36-bit mode.

FIG. 51 is a state machine diagram for a 48-bit mode.

FIG. 52 shows recommended N and expected CTS of 36 bits/pixel for 32 kHzand multiples.

FIG. 53 shows recommended N and expected CTS of 36 bits/pixel for 44.1kHz and multiples.

FIG. 54 shows recommended N and expected CTS of 36 bits/pixel for 48 kHzand multiples.

FIG. 55 shows recommended N and expected CTS of 48 bits/pixel for 32 kHzand multiples.

FIG. 56 shows recommended N and expected CTS of 48 bits/pixel for 44.1kHz and multiples.

FIG. 57 shows recommended N and expected CTS of 48 bits/pixel for 48 kHzand multiples.

REFERENCE NUMERALS

41 HD recorder, 42 display, 43 cable, 51 recording/reproduction section,52 codec, 53 HDMI(R) source, 54 HD, 61 HDMI(R) sink, 62 displaycontroller, 63 display section, 71 source signal processor, 72transmitter, 72A to 72C encoder/serializer, 81 receiver, 81A to 81Crecovery/decoder, 82 sink signal processor, 101 main image processor,102 sub-signal addition section, 103 sub-signal processor, 104sub-signal-related information insertion section, 105 sub-signalreception capability/incapability determination section, 106 number ofsub-signal assignment bits determination section, 107 sub-signal frameinformation transmission controller, 108 deep-color-mode determinationsection, 121 FIFO memory, 122 sub-signal presence/absence determinationsection, 123 separator, 124 main image processor, 125 main image memory,126 sub-signal processor, 127 sub-signal memory, 201 bus, 202 CPU, 203ROM, 204 RAM, 205 EEPROM, 206 input/output interface, 301 HDMI cable,310 video recording/reproduction apparatus (source-side apparatus), 311recording/reproduction section, 312 video processor, 314 audioprocessor, 315 controller, 316 tuner, 320 HDMI transmission processor,321 multiplexing circuit, 322 HDCP encryptor, 323 transmissionprocessor, 324 HDMI terminal, 330 television receiver (sink-sideapparatus), 331 video selector/combiner, 332 video processor, 333display processor, 334 audio processor, 335 output processor, 336controller, 340 HDMI transmission processor, 341 HDMI terminal, 342transmission processor, 343 HDCP decryptor, 344 demultiplexing circuit,351 to 354 speaker, 360 display panel

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will now be described below withreference to the drawings.

FIG. 1 shows an example of the configuration of an embodiment of an AV(Audio Visual) system to which the present invention is applied.

In FIG. 1, an AV system includes an HD (Hard Disk) recorder 41 and adisplay 42, with the HD recorder 41 and the display 42 being connectedto each other via a cable 43 for HDMI(R).

The HD recorder 41 includes a recording/reproduction section 51, a codec52, an HDMI(R) source 53, and an HD 54, and records and reproduces datato and from the HD 54.

That is, the recording/reproduction section 51 records, in the HD 54,encoded data obtained by encoding data of a baseband image and audioaccompanied therewith, which is supplied from the codec 52, by using anMPEG (Moving Picture Experts Group) scheme or the like. Furthermore, therecording/reproduction section 51 reproduces (reads) the encoded datafrom the HD 54 and supplies it to the codec 52.

The codec 52 decodes the encoded data supplied from therecording/reproduction section 51 into data of a baseband image andaudio by using an MPEG scheme or the like, and supplies the data of thebaseband image and audio to the HDMI (R) source 53 and an externalapparatus (not shown).

Furthermore, the codec 52 encodes, for example, data of a baseband imageand audio, which is supplied from an external apparatus (not shown),into encoded data, and supplies the encoded data to therecording/reproduction section 51.

The HDMI(R) source 53 transmits the data of the baseband image andaudio, which is supplied from the codec 52, to the display 42 via thecable 43 in one direction in accordance with communication compliantwith HDMI(R).

The display 42 includes an HDMI(R) sink 61, a display controller 62, anda display section 63, and displays an image.

That is, the HDMI(R) sink 61 receives data of a baseband image andaudio, which is transmitted from the HDMI(R) source 53 of the HDrecorder 41 connected via the cable 43 in one direction in accordancewith communication in compliance with HDMI(R), and supplies the data ofthe image among the received data to the display controller 62. The dataof audio, which is received by the HDMI(R) sink 61, is supplied to, forexample, a speaker (not shown) incorporated in the display 42 and isoutput therefrom.

The display controller 62 controls (drives) the display section 63 onthe basis of the data of the baseband image supplied from the HDMI(R)sink 61, so that the display section 63 displays a corresponding image.

The display section 63 is constituted by, for example, a CRT (CathodeRay Tube), an LCD (Liquid Crystal Display), an organic EL (ElectroLuminescence), or the like, and displays an image under the control ofthe display controller 62.

In the AV system of FIG. 1, which is configured as described above, forexample, when a user operates the HD recorder 41 so as to reproduce theHD 54, the recording/reproduction section 51 reproduces encoded datafrom the HD 54 and supplies it to the codec 52.

The codec 52 decodes the encoded data supplied from therecording/reproduction section 51 into baseband image and audio data,and supplies the baseband image and audio data to the HDMI(R) source 53.

On the basis of communication in compliance with HDMI (R), the HDMI(R)source 53 transmits the baseband image and audio data supplied from thecodec 52 to the display 42 in one direction via the cable 43.

In the display 42, on the basis of communication in compliance withHDMI(R), the HDMI(R) sink 61 receives the baseband image and audio datatransmitted in one direction from the HDMI(R) source 53 of the HDrecorder 41 connected via the cable 43, and supplies the image dataamong the received data to the display controller 62 and supplies theaudio data to a speaker (not shown).

The display controller 62 controls the display section 63 on the basisof the image data supplied from the HDMI(R) sink 61, so that acorresponding image is displayed on the display section 63.

FIG. 2 shows an example of the configuration of the HDMI(R) source 53and the HDMI(R) sink 61.

In a valid image period (hereinafter referred to as an “active videoperiod” as appropriate) that is a period in which a horizontal blankingperiod and a vertical blanking period are excluded from the period fromone vertical synchronization signal to the next vertical synchronizationsignal, the HDMI(R) source 53 transmits a differential signalcorresponding to pixel data of an uncompressed image for one screen tothe HDMI(R) sink 61 in one direction through a plurality of channels. Inthe horizontal blanking period or in the vertical blanking period, theHDMI(R) source 53 transmits differential signals corresponding to audiodata accompanied with an image, a control packet, and other auxiliarydata to an HDMI(R) sink in one direction through a plurality ofchannels.

That is, the HDMI(R) source 53 includes a source signal processor 71 anda transmitter 72.

Data of an uncompressed baseband image (Video) and audio (Audio) issupplied to the source signal processor 71 from the codec 52 (FIG. 1) orthe like. The source signal processor 71 performs necessary processingon the image and audio data supplied thereto and supplies the data tothe transmitter 72. Furthermore, the source signal processor 71transmits and receives information for control and information thatinforms status (Control/Status) to and from the transmitter 72 asnecessary.

The transmitter 72 converts the pixel data of the image supplied fromthe source signal processor 71 into a corresponding differential signal,and serially transmits the differential signal to the HDMI(R) sink 61connected via the cable 43 in one direction through three TMDS channels#0, #1, and #2.

Furthermore, the transmitter 72 converts audio data accompanied with anuncompressed image, a control packet, and other auxiliary data, whichare supplied from the source signal processor 71, and control data suchas a vertical synchronization signal (VSYNC) and a horizontalsynchronization signal (HSYNC), into corresponding differential signals,and serially transmits the differential signals to the HDMI(R) sink 61connected via the cable 43 in one direction through three TMDS channels#0, #1, and #2.

Furthermore, the transmitter 72 transmits a pixel clock synchronizedwith pixel data to be transmitted through three TMDS channels #0, #1,and #2 to the HDMI(R) sink 61 connected via the cable 43 in a TMDS clockchannel.

In the active video period, the HDMI(R) sink 61 receives thedifferential signal corresponding to the pixel data transmitted in onedirection from the HDMI(R) source 53 through a plurality of channels,and receives a differential signal corresponding to the auxiliary dataand the control data, which is transmitted in one direction from theHDMI(R) source through a plurality of channels in the horizontalblanking period and in the vertical blanking period.

That is, the HDMI(R) sink 61 includes a receiver 81 and a sink signalprocessor 82.

The receiver 81 receives, through the TMDS channels #0, #1, and #2, adifferential signal corresponding the pixel data and a differentialsignal corresponding to the auxiliary data and the control data, whichare transmitted in one direction from the HDMI(R) source 53 connectedvia the cable 43 in synchronization with a pixel clock transmittedthrough a TMDS clock channel from the HDMI(R) source 53 in a similarmanner.

Furthermore, the receiver 81 converts the differential signal intocorresponding pixel data, auxiliary data, and control data, and suppliesthem to the sink signal processor 82 as necessary.

The sink signal processor 82 performs necessary processing on the datasupplied from the receiver 81, and supplies the data to the displaycontroller 62 and the like. In addition, the sink signal processor 82transmits and receives information for control and information thatinforms status (Control/Status) to and from the receiver 81 asnecessary.

The transmission channels of HDMI(R) include three TMDS channels #0 to#2 serving as transmission channels for serially transmitting pixeldata, auxiliary data, and control data in synchronization with a pixelclock in one direction from the HDMI(R) source 53 to the HDMI(R) sink61, and transmission channels called DDC (Display Data Channel) and CEClines in addition to a TMDS clock channel serving as a transmissionchannel for transmitting the pixel clock.

The DDC is used for the HDMI(R) source 53 to read E-EDID (EnhancedExtended Display Identification Data) from the HDMI(R) sink 61 connectedvia the cable 43.

That is, the HDMI(R) sink 61 has an EDID ROM (Read Only Memory) (notshown) storing E-EDID that is capability information regarding its owncapability (configuration/capability) in addition to the receiver 81.The HDMI(R) source 53 reads, via the DDC, the E-EDID of the HDMI(R) sink61 from the HDMI(R) sink 61 connected via the cable 43. On the basis ofthe E-EDID, the HDMI(R) source 53 recognizes capability and setting ofthe HDMI(R) sink 61, that is, for example, the format (the profile) ofan image that (an electronic apparatus having) the HDMI(R) sink 61supports (for example, RGB (Red, Green, Blue), YC_(B)C_(R) 4:4:4,YC_(B)C_(R) 4:2:2).

The HDMI(R) source 53 can also store E-EDID and transmit the E-EDID tothe HDMI(R) sink 61 as necessary in the same manner as for the HDMI(R)sink 61.

The CEC line is used to perform bidirectional communication of data forcontrol between the HDMI(R) source 53 and an HDMI(R) link 2.

FIG. 3 shows an example of the configuration of the transmitter 72 andthe receiver 81 illustrated in FIG. 2.

The transmitter 72 includes three encoders/serializers 72A, 72B, and 72Ccorresponding to three TMDS channels #0 to #2, respectively. Then, eachof the encoders/serializers 72A, 72B, and 72C encodes pixel data,auxiliary data, and control data supplied thereto, converts them fromparallel data into serial data, and transmits it using a differentialsignal.

At this point, when the pixel data has, for example, three components ofR, G, and B, the B component is supplied to the encoder/serializer 72A,the G component is supplied to the encoder/serializer 72B, and the Rcomponent is supplied to the encoder/serializer 72C.

Examples of auxiliary data include audio data and a control packet. Thecontrol packet is supplied to, for example, the encoder/serializer 72A,and the audio data is supplied to, for example, the encoders/serializers72B and 72C.

Furthermore, the control data includes a 1-bit vertical synchronizationsignal (VSYNC), a 1-bit horizontal synchronization signal (HSYNC), andcontrol bits CTL0, CTL1, CTL2, and CTL3, each of which is 1 bit. Thevertical synchronization signal and the horizontal synchronizationsignal are supplied to the encoder/serializer 72A, the control bits CTL0and CTL1 are supplied to the encoder/serializer 72B, and the controlbits CTL2 and the CTL3 are supplied to the encoder/serializer 72C.

The encoder/serializer 72A transmits the B component of the pixel data,the vertical synchronization signal and the horizontal synchronizationsignal, and auxiliary data, which are supplied thereto, at timedivision.

That is, the encoder/serializer 72A converts the B component of thepixel data supplied thereto into parallel data in units of 8 bits, whichis a fixed number of bits. Furthermore, the encoder/serializer 72Aencodes the parallel data so as to convert the data into serial data,and transmits it through the TMDS channel #0.

Furthermore, the encoder/serializer 72A encodes 2-bit parallel data ofthe vertical synchronization signal and the horizontal synchronizationsignal, which are supplied thereto, so as to convert the data intoserial data, and transmits it through the TMDS channel #0.

Furthermore, the encoder/serializer 72A converts the auxiliary datasupplied thereto into parallel data in units of 4 bits. Then, theencoder/serializer 72A encodes the parallel data so as to convert thedata into serial data, and transmits it through the TMDS channel #0.

The encoder/serializer 72B transmits the G component of the pixel data,the control bits CTL0 and CTL1, and the auxiliary data, which aresupplied thereto, at time division.

That is, the encoder/serializer 72B converts the G component of thepixel data supplied thereto into parallel data in units of 8 bits, whichis a fixed number of bits. Furthermore, the encoder/serializer 72Bencodes the parallel data so as to convert the data into serial data,and transmits it through the TMDS channel #1.

Furthermore, the encoder/serializer 72B encodes 2-bit parallel data ofthe control bits CTL0 and CTL1 supplied thereto so as to convert thedata into serial data, and transmits it through the TMDS channel #1.

Furthermore, the encoder/serializer 72B converts the auxiliary datasupplied thereto into parallel data in units of 4 bits. Then, theencoder/serializer 72B encodes parallel data so as to convert the datainto serial data, and transmits it through the TMDS channel #1.

The encoder/serializer 72C transmits the R component, the control bitsCTL2 and CTL3, and the auxiliary data, which are supplied thereto, attime division.

That is, the encoder/serializer 72C converts the R component of thepixel data supplied thereto into parallel data in units of 8 bits, whichis a fixed number of bits. Furthermore, the encoder/serializer 72Cencodes the parallel data so as to convert the data into serial data,and transmits it through the TMDS channel #2.

Furthermore, the encoder/serializer 72C encodes the 2-bit parallel dataof the control bits CTL2 and CTL3 supplied thereto so as to convert thedata into serial data, and transmits it through the TMDS channel #2.

Furthermore, the encoder/serializer 72C converts the auxiliary datasupplied thereto into parallel data in units of 4 bits. Then, theencoder/serializer 72C encodes the parallel data so as to convert thedata into serial data, and transmits it through the TMDS channel #2.

The receiver 81 includes three recovery/decoders 81A, 81B, and 81Ccorresponding to the three TMDS channels #0 to #2, respectively. Each ofthe recovery/decoders 81A, 81B, and 81C receives pixel data, auxiliarydata, and control data, which are transmitted using a differentialsignal through the TMDS channels #0 to #2. Furthermore, each of therecovery/decoders 81A, 81B, and 81C converts the pixel data, theauxiliary data, and the control data from serial data into paralleldata, decodes them, and outputs them.

That is, the recovery/decoder 81A receives the B component of the pixeldata, the vertical synchronization signal, the horizontalsynchronization signal, and the auxiliary data, which are transmittedusing a differential signal through the TMDS channel #0. Then, therecovery/decoder 81A converts the B component of the pixel data, thevertical synchronization signal, the horizontal synchronization signal,and the auxiliary data from serial data into parallel data, decodesthem, and outputs them.

The recovery/decoder 81B receives the G component of the pixel data, thecontrol bits CTL0 and CTL1, and the auxiliary data, which aretransmitted using a differential signal through the TMDS channel #1.Then, the recovery/decoder 81B converts the G component of the pixeldata, the control bits CTL0 and CTL1, the auxiliary data from serialdata into parallel data, decodes them, and outputs them.

The recovery/decoder 81C receives the R component of the pixel data, thecontrol bits CTL2 and CTL3, and the auxiliary data, which aretransmitted using a differential signal through the TMDS channel #2.Then, the recovery/decoder 81C converts the R component of the pixeldata, the control bits CTL2 and CTL3, and the auxiliary data from serialdata into parallel data, decodes them, and outputs them.

FIG. 4 shows an example of a transmission period in which various kindsof transmission data are transmitted through the three TMDS channels #0to #2 of HDMI(R).

FIG. 4 shows a transmission period of various kinds of transmission datawhen a progressive image of a 720 . . . × . . . 480 matrix of pixels istransmitted through the TMDS channels #0 to #2.

In a video field in which transmission data is transmitted through thethree TMDS channels #0 to #2 of HDMI(R), there are three kinds ofperiods, that is, a video data period, a data island period, and acontrol period, according to the type of transmission data.

At this point, the video field is a period from (the rise edge (activeedge)) of one vertical synchronization signal to the next verticalsynchronization signal, and is divided into a horizontal blanking period(horizontal blanking), a vertical blanking period (vertical blanking),and an active video period (Active Video), which is a period in whichthe horizontal blanking period and the vertical blanking period areexcluded from the video field.

A video data period (the portion shaded with left upward (rightdownward) diagonal lines in FIG. 4) is assigned to an active videoperiod. In the video data period, pixel (active pixels) data of anuncompressed image for one screen is transmitted.

A data island period (the portion shaded with right upward (leftdownward) diagonal lines in FIG. 4) and a control period (the portionshaded with lines in the vertical direction in FIG. 4) are assigned to ahorizontal blanking period and a vertical blanking period. In the dataisland period and the control period, auxiliary data is transmitted.

That is, the data island period is assigned to portions of thehorizontal blanking period and the vertical blanking period. In the dataisland period, data that is not related to control, for example, packetsof audio data, among the auxiliary data, is transmitted.

The control period is assigned to the other portions of the horizontalblanking period and the vertical blanking period. In the control period,data that is related to control, for example, the verticalsynchronization signal, the horizontal synchronization signal, controlpackets, and the like, among the auxiliary data, is transmitted.

At this point, in the current HDMI(R), that is, in “High-DefinitionMultimedia Interface Specification Version 1.2a”, Dec. 14, 2005, whichis the up-to-date specification of HDMI(R), the frequency of the pixelclock transmitted through a TDMS clock channel (FIG. 2) is, for example,165 MHz. In this case, the transmission rate in the data island periodis approximately 500 Mbps.

As described above, in both the data island period and the controlperiod, the auxiliary data is transmitted, and the distinctiontherebetween is made possible using the control bits CTL0 and CTL1.

FIG. 5 shows the relationship among the control bits CTL0 and CTL1, adata island period, and a control period.

The control bits CTL0 and CTL1 can represent, for example, two states, adevice enable state and a device disable state, as shown in the firstarea from the top in FIG. 5. In the first area from the top in FIG. 5,the device enable state is represented using an H (High) level and thedevice disable state is represented using an L (Low) level.

The control bits CTL0 and CTL1 enter a device disable state in the dataisland period and enter a device enable state in the control period.This makes it possible to distinguish between the data island period andthe control period.

Then, in the data island period in which the control bits CTL0 and CTL1become an L level denoting a device disable state, as shown from thesecond area from the top in FIG. 5, data that is not related to control,for example, audio data, among the auxiliary data, is transmitted.

On the other hand, in the control period in which the control bits CTL0and CTL1 become an H level denoting a device enable state, as shown fromthe third area from the top in FIG. 5, data that is related to control,for example, a control packet and a preamble, among the auxiliary data,is transmitted.

In addition, in the control period, as shown from the fourth area fromthe top in FIG. 5, a vertical synchronization signal and a horizontalsynchronization signal are also transmitted.

Next, a description will be given, with reference to FIG. 6, of thetransmission of pixel data, which is defined in the current HDMI(R),that is, “High-Definition Multimedia Interface Specification Version1.2a”, Dec. 14, 2005, which is the up-to-date specification of HDMI(R).

FIG. 6 is a timing chart showing the timing of transmission of pixeldata of an image transmitted in a video data period of the currentHDMI(R).

In the current HDMI(R), pixel data of an image of three formats of RGB4:4:4, YC_(B)C_(R) 4:4:4, and YC_(B)C_(R) 4:2:2 can be transmittedthrough the TMDS channels #0 to #2. In the following, a description willbe given using, for example, RGB 4:4:4 as an example among theabove-described three formats.

In HDCP, a technology for preventing copying of content, which isadopted in HDMI(R), data is scrambled in units of 8 bits. For thisreason, through one TMDS channel, data is transmitted in units of afixed number of bits, that is, in units of 8 bits for the object ofprocessing in HDCP, per clock of a pixel clock.

In the manner described above, through one TMDS channel, since 8-bitdata is transmitted per clock of a pixel clock, it is possible for thethree TMDS channels #0 to #2 to transmit 24-bit data per clock.

Therefore, in the current HDMI(R), a 24-bit image as an image of RGB4:4:4, in which each of the R component, the G component, and the Bcomponent of each pixel is 8 bits, is transmitted through the three TMDSchannels #0 to #2.

That is, in the current HDMI(R), as shown in FIG. 6, per clock of apixel clock, the 8-bit B component among the pixel data of one pixel ofa 24-bit image is transmitted through TMDS channel #0, the 8-bit Gcomponent is transmitted through TMDS channel #1, and the 8-bit Rcomponent is transmitted through TMDS channel #2.

As described above, in recent years, there has been an increasing demandfor transmitting an image having a higher resolution, that is, a highresolution image formed of a large number of bits of pixel data suchthat each of the R component, the G component, and the B component is 10bits or 12 bits, which is larger than 8 bits.

In the current HDMI(R), as described above, the transmission of a 24-bitimage is performed in such a manner that 8-bit data is transmitted perclock of a pixel clock through one TMDS channel. Therefore, simply, thetransmission of a high resolution image can be performed by transmittingdata of, for example, 10 bits or 12 bits, which is larger than 8 bits,per clock of a pixel clock through one TMDS channel.

However, as described above, in HDCP, a technology for preventingcopying of content, which is adopted in HDMI(R), data is scrambled inunits of 8 bits. Therefore, when data transmission is to be performed inunits of bits other than 8 bits per clock of a pixel clock through oneTMDS channel, it is difficult to apply HDCP. As a result, it isdifficult to perform data transmission in compliance with HDMI(R).

Therefore, in the current HDMI(R), as described above, a pixel clock ofthe frequency of 165 MHz is employed. By employing, as this pixel clock,a pixel clock having a higher frequency, a high resolution image canalso be transmitted while HDCP can be applied by transmitting data infixed units of 8 bits per clock of a pixel clock through one TMDSchannel.

It is possible to transmit a 10-bit high resolution image (hereinafteralso referred to as a “30 bit (=10 bits× . . . 3) image” as appropriate)such that each of the R component, the G component, and the B componentis 10 bits by setting, for example, the frequency of the pixel clock to5/4 times 165 MHz when a 24-bit image is to be transmitted.

FIG. 7 is a timing chart showing the timing of transmission of pixeldata when a 30-bit image is to be transmitted in a video data period ofHDMI(R).

Transmission of a 30-bit image is common to the transmission of a 24-bitimage in that the B component is transmitted through the TMDS channel#0, the G component is transmitted through the TMDS channel #1, and theR component is transmitted through the TMDS channel #2, and in that8-bit data is transmitted per clock of a pixel clock.

However, the transmission of a 30-bit image differs from thetransmission of a 24-bit image in the following points. In thetransmission of a 24-bit image, since one component (the R component,the G component, or the B component) of one pixel is 8 bits, the 8-bitcomponents are transmitted in one clock of the pixel clock. Incomparison, in the transmission of a 30-bit image, since one componentof one pixel is 10 bits, the 10-bit components are transmitted over aplurality of clocks of the pixel clock.

That is, the 10-bit components from the LSB (Least Significant Bit) tothe MSB (Most Significant Bit) are denoted as b0 to b9. The R component,the G component, and the B component of the i-th pixel in the rasterscan sequence, of the pixel constituting the image are denoted as R #i−1component, G #i−1 component, and B #i−1 component, respectively, and thej-th clock will be referred to as a clock #j−1 by using a particularclock (pulse) of a pixel clock as a reference.

In this case, as shown in FIG. 7, regarding the B component, thelow-order 8 bits b0 to b7 among the 10-bit B#0 components aretransmitted in clock #0, and a total of 8 bits of the remaininghigh-order 2 bits b8 and b9 among the 10-bit B#0 component and thelow-order 6 bits b0 to b5 among the 10-bit B#1 components of the nextpixel are transmitted in clock #1.

Furthermore, a total of 8 bits of the remaining high-order 4 bits b6 tob9 among the 10-bit B#1 components and the low-order 4 bits b0 to b3among the 10-bit B#2 components of the next pixel are transmitted inclock #2. A total of 8 bits of the remaining high-order 6 bits b4 to b9among the 10-bit B#2 components, and the low-order 2 bits b0 and b1among the 10-bit B#3 components of the next pixel are transmitted inclock #3.

Then, the remaining high-order 8 bits b2 to b9 among the 10-bit B#3components are transmitted in clock #4.

In the manner described above, B#0 to B#3 components, which are Bcomponents of 4 pixels, are transmitted in 5 clocks of clocks #0 to #4,and hereinafter, the B components of 4 pixels are transmitted in 5clocks in a similar manner.

As illustrated in FIG. 6, regarding a 24-bit image, 8-bit B componentsare transmitted in 1 clock, and regarding a 30-bit image, 10-bit Bcomponents for 4 pixels are transmitted in 5 clocks. Therefore,regarding a 30-bit image, by transmitting the B component by using apixel clock of a frequency 5/4 times the frequency of the 24-bit image,one frame of the 30-bit image can be transmitted in the same period oftime as for one frame of the 24-bit image.

Components other than the B component of the 30-bit image, that is, theG component and the R component, are also transmitted in the same manneras for the B component.

Furthermore, in the transmission of a 30-bit image, the transmission ofpixel data of 4 pixels in 5 clocks forms a transmission unit, and thetransmission unit is repeated. If the clock in the transmission unit isreferred to as a phase, the transmission unit of the 30-bit image iscomposed of five phases.

Next, by increasing the frequency of the pixel clock, it is possible totransmit a high resolution image having a resolution higher than that ofa 30-bit image, that is, a high resolution image in which the Rcomponent, the G component, and the B component are each, for example,12 bits (hereinafter also referred to as a 36-bit (=12 bits× . . . 3)image as appropriate), and a high resolution image in which the Rcomponent, the G component, and the B component are each, for example,16 bits (hereinafter also referred to as a 48-bit (=16 bits× . . . 3)image as appropriate).

More specifically, for example, by setting the frequency of the pixelclock to be 3/2 times 165 MHz in the case of transmitting a 24-bitimage, it is possible to transmit a 36-bit image in which the Rcomponent, G component, and B component are each 12 bits.

FIG. 8 is a timing chart showing the timing of transmission of pixeldata when a 36-bit image is to be transmitted in a video data period ofHDMI(R).

The transmission of a 36-bit image is common to the transmission of a24-bit image in that the B component is transmitted through the TMDSchannel #0, the G component is transmitted through the TMDS channel #1,and the R component is transmitted through the TMDS channel #2, and inthat 8-bit data is transmitted per clock of a pixel clock.

However, in the transmission of a 24-bit image, since one component ofone pixel is 8 bits, the 8-bit component is transmitted in one clock ofa pixel clock. In comparison, in the transmission of a 36-bit image,since one component of one pixel is 12 bits, the transmission of a36-bit image differs from the transmission of a 24-bit image in that the12-bit component is transmitted over a plurality of pixel clocks.

That is, if the 12-bit components from the LSB to the MSB are denoted asb0 to b11 regarding the B components, as shown in FIG. 8, the low-order8 bits b0 to b7 among the 12-bit B#0 components are transmitted in clock#0. A total of 8 bits of the remaining high-order 4 bits b8 to b11 amongthe 12-bit B#1 components, and the low-order 4 bits b0 to b3 of the nextpixel are transmitted in clock #1.

Then, the remaining high-order 8 bits b4 to b11 among the 12-bit B#1components are transmitted in clock #2.

In the manner described above, B#0 and B#1 components, which are Bcomponents of two pixels, are transmitted in 3 clocks of clocks #0 to#2, and hereinafter, the B components of two pixels are transmitted in 3clocks in a similar manner.

As illustrated in FIG. 6, regarding the 24-bit image, 8-bit B componentsare transmitted in one clock, and regarding the 36-bit image, 12-bit Bcomponents for two pixels are transmitted in 3 clocks. Therefore,regarding the 36-bit image, by transmitting the B components by using apixel clock of a frequency 3/2 times that of the 24-bit image, it ispossible to transmit one frame of the 36-bit image in the same period oftime as for one frame of the 24-bit image.

Components other than the B component of the 36-bit image, that is, theG component and the R component, are transmitted in the same manner asfor the B component.

Furthermore, in the transmission of a 36-bit image, the transmission ofpixel data of 2 pixels in 3 clocks forms a transmission unit, and thetransmission unit is repeated. Therefore, the transmission unit of the36-bit image is composed of three phases.

Next, the transmission of a 48-bit image in which the R component, the Gcomponent, and the B component are each 16 bits can be performed bysetting the frequency of the pixel clock to be two times 165 MHz in thecase of transmitting a 24-bit image.

FIG. 9 is a timing chart showing the timing of transmission of pixeldata when a 48-bit image is to be transmitted in a video data period ofHDMI(R).

The transmission of a 48-bit image is common to the transmission of a24-bit image in that the B component is transmitted through TMDS channel#0, the G component is transmitted through TMDS channel #1, the Rcomponent is transmitted through TMDS channel #2, and in that 8-bit datais transmitted per clock of a pixel clock.

The transmission of a 48-bit image differs from the transmission of a24-bit image as follows. In the transmission of a 24-bit image, sinceone component of one pixel is 8 bits, the 8-bit component is transmittedin one clock of a pixel clock. In comparison, in the transmission of a48-bit image, since one component of one pixel is 16 bits, the 16-bitcomponent is transmitted over a plurality of clocks of the pixel clock.

That is, if 16-bit components from the LSB to the MSB are denoted as b0to b15, as shown in FIG. 9, regarding the B components, the low-order 8bits b0 to b7 among the 16-bit B#0 component are transmitted in clock#0, and the remaining high-order 8 bits b8 to b15 among the 16-bit B#0component are transmitted in clock #1.

In the manner described above, the B#0 component, which is the Bcomponent of one pixel, is transmitted in two clocks of clocks #0 and#1, and hereinafter, the B component of one pixel is transmitted in twoclocks in a similar manner.

As illustrated in FIG. 6, regarding the 24-bit image, 8-bit B componentsare transmitted in one clock, and regarding the 48-bit image, 16-bit Bcomponents for one pixel are transmitted in two clocks. Therefore,regarding the 48-bit image, by transmitting the B component by using apixel clock of a frequency two times that of the 24-bit image, it ispossible to transmit one frame of the 48-bit image in the same period oftime as that of one frame of the 24-bit image.

Components other than the B component of the 48-bit image, that is, theG component and the R component, are also transmitted in the same manneras for the B component.

Furthermore, in the transmission of a 48-bit image, the transmission ofpixel data of 1 pixel in 2 clocks forms a transmission unit, and thetransmission unit is repeated. Therefore, the transmission unit of the48-bit image is composed of two phases.

In the manner described above, by adjusting the frequency of the pixelclock so as to be, for example, 5/4 times, 3/2 times, and 2 times thatin the case of transmitting a 24-bit image, the transmission of pixeldata in which 10 bits, 12 bits, and 16 bits, which is greater than 8bits, which is a fixed number of bits that are transmitted per clock ofa pixel clock through the TMDS channel of the current HDMI(R), areassigned to each component, that is, the transmission of a highresolution image, such as a 30-bit image, a 36-bit image, or a 48-bitimage, can be performed by using the TMDS channel of the current HDMI(R)as it is.

Therefore, when it is considered that the transmission of a 24-bit imageis performed in the current HDMI(R), by adjusting the frequency of thepixel clock, the transmission of pixel data in which a number of bits,which is greater than 8 bits, which is a fixed number of bits that aretransmitted per clock of a pixel clock through the TMDS channel of thecurrent HDMI(R), is assigned to each component, that is, for example,the transmission of a high resolution image such as a 30-bit image, a36-bit image, or a 48-bit image, in addition to a 24-bit image, can beperformed by using the TMDS channel of the current HDMI(R) as it is.

If a communication interface capable of performing the transmission of ahigh resolution image in addition to a 24-bit image by using the TMDSchannel of the current HDMI(R) as it is referred to as, in particular,deep color HDMI(R), in the case in which, for example, an HDMI(R) sourcein compliance with deep color HDMI(R) is to transmit a high resolutionimage, first, it is necessary to recognize whether or not a HDMI(R) sinkwith which communication is performed supports (complies with) deepcolor HDMI(R).

At this point, whether or not the HDMI(R) sink complies with deep colorHDMI(R) can be described (contained) in E-EDID that is capabilityinformation on the capability of the HDMI(R) sink.

FIG. 10 shows VSDB (Vender Specific Definition Bit) in E-EDID.

In the current HDMI(R), bits #4, #5, and #6, which are fifth, sixth, andseventh bits from the LSB of byte #6 of VSDB are unused (Reserved). InFIG. 10, bits Suport_(—)30 bit, Suport_(—)36 bit, and Suport_(—)48 bitare assigned to bits #4, #5, and #6, respectively.

All the bit Suport_(—)30 bit assigned to bit #4 of byte #6 of VSDB, thebit Suport_(—)36 bit assigned to bit #5, and the bit Suport_(—)48 bitassigned to bit #6 are set to, for example, 0 when the HDMI(R) sink doesnot support a high resolution image, that is, when HDMI(R) sink supportsonly a 24-bit image.

When the HDMI(R) sink supports only a 30-bit image among high resolutionimages, only the bit Suport_(—)30 bit is set to 1. When the HDMI(R) sinksupports only a 30-bit image and a 36-bit image among the highresolution images, only the bit Suport_(—)36 bit is set to 1.Furthermore, when the HDMI(R) sink supports all of the 30-bit image, the36-bit image, and the 48-bit image, only the bit Suport_(—)48 bit is setto 1.

In the manner described above, as a result of describing whether or notthe HDMI(R) sink complies with deep color HDMI(R) in the VSDB of theE-EDID, it is possible for the HDMI(R) source to recognize whether ornot the HDMI(R) sink supports a high resolution image by reading E-EDIDfrom the HDMI(R) sink and by referring to the VSDB of the E-EDID.Furthermore, when the HDMI(R) sink supports a high resolution image, itis possible to recognize which one of the 30-bit image, the 36-bitimage, and the 48-bit image the HDMI(R) sink supports.

The bits Suport_(—)30 bit, Suport_(—)36 bit, and Suport_(—)48 bit shownin FIG. 10 can also be described in the VSDB of the E-EDID of theHDMI(R) source.

Next, exchanging of E-EDID between the HDMI(R) source and the HDMI(R)sink is performed at a specific timing, such as when the HDMI(R) sourceand the HDMI(R) sink are connected to each other or when the powersupply of the HDMI(R) source or the HDMI(R) sink is turned on, and isnot performed in a periodic manner.

On the other hand, when the HDMI(R) source and the HDMI(R) sink supporta high resolution image, there are cases in which a 24-bit image istransmitted from the HDMI(R) source to the HDMI(R) sink or a highresolution image is transmitted, and in the case of transmitting a highresolution image, there is a case in which a 30-bit image, a 36-bitimage, or a 48-bit image is transmitted.

As illustrated in FIG. 4, the transmission of an image is performed in avideo data period assigned to an active video period (Active Video) of avideo field. Therefore, it is preferable that the HDMI(R) sink canrecognize which one of the 24-bit image, the 30-bit image, the 36-bitimage, and the 48-bit image the image transmitted in the video dataperiod is for each video field containing the video data period.

In this case, it is necessary to transmit for each video field, from theHDMI(R) source to the HDMI(R) sink, information (hereinafter alsoreferred to as a “deep color mode” as appropriate) indicating which oneof the 24-bit image, the 30-bit image, the 36-bit image, and the 48-bitimage the image transmitted in the video data period contained in thevideo field is.

At this point, as information transmitted for each video field from theHDMI(R) source into the HDMI(R) sink, there is a general control packettransmitted in the control period illustrated in FIG. 4 within thevertical blanking period.

Therefore, the deep color mode can be contained in a general controlpacket, so that the deep color mode is transmitted from the HDMI(R)source to the HDMI(R) sink for each video field.

FIG. 11 shows the format of a general control packet.

The general control packet has a packet header (General Control PacketHeader) and a subpacket (General Control Subpacket). The upper area ofFIG. 11 shows a packet header, and the lower area of FIG. 11 shows asubpacket.

In the current HDMI(R), it is stipulated that bits #0, #1, and #2, whichare first, second, and third bits from the LSB of byte #SB1 of asubpacket (the lower area of FIG. 11) of a general control packet, areunused and set to 0. In FIG. 11, bits CD0, CD1, and CD2 indicating adeep color mode are assigned to the bits #0, #1, and #2, respectively.

FIG. 12 shows the relationship between bits CD0, CD1, and CD2 of byte#SB1 of a subpacket, and an image transmitted in a video data period(FIG. 4) contained in a video field containing a control period (FIG. 4)in which a general control packet having a subpacket is transmitted.

When the HDMI(R) sink does not support a high resolution image (ColorDepth not indicated), all the bits CD0, CD1, and CD2 indicating a deepcolor mode are set to 0 in the same manner as in the current HDMI(R).

Furthermore, when the HDMI(R) sink supports a high resolution image, inthe case that an image transmitted in the video data period is a 24-bitimage, the bits CD0, CD1, and CD2 indicating a deep color mode are setto, for example, 0, 0, and 1, respectively. When the image transmittedin the video data period is a 30-bit image, the bits CD0, CD1, and CD2indicating a deep color mode are set to, for example, 1, 0, and 1,respectively.

Furthermore, when the image transmitted in the video data period is a36-bit image, the bits CD0, CD1, and CD2 indicating a deep color modeare set to, for example, 0, 1, and 1, respectively. When the imagetransmitted in the video data period is a 48-bit image, all bits CD0,CD1, and CD2 indicating a deep color mode are set to, for example, 1.

In the manner described above, the HDMI(R) source transmits the bitsCD0, CD1, and CD2 indicating a deep color mode, with the bits beingcontained in the general control packet, in the control period of thevideo field. As a result, it is possible for the HDMI(R) sink torecognize which one of the 24-bit image, the 30-bit image, the 36-bitimage, and the 48-bit image the image transmitted in the video dataperiod of the video field is.

In the current HDMI(R), the bits #4, #5, and #6 of the fifth, sixth, andseventh bits from the LSB of byte #SB1 of a subpacket (the lower area ofFIG. 11) of the general control packet shown in FIG. 11 are assumed tobe unused and set to 0. In FIG. 11, the bits PP0, PP1, and PP2indicating a phase are assigned to the bits #4, #5, and #6,respectively.

That is, when a 30-bit image, a 36-bit image or a 48-bit image is to betransmitted, a phase exists, as illustrated in FIGS. 7 to 9,respectively. In the bits PP0, PP1, and PP2 of byte #SB1 of thesubpacket, values indicating the phase of the pixel data that istransmitted finally among the pixel data of an image transmitted in thevideo data period contained in the video field containing the controlperiod in which the general control packet having the subpacket istransmitted are set.

Next, a description will be given, with reference to the flowcharts inFIGS. 13 and 14, of the operation of the HDMI(R) source 53 and theHDMI(R) sink 61 when the HDMI(R) source 53 and the HDMI(R) sink 61 inFIG. 2 complies with deep color HDMI(R).

First, a description will be given below, with reference to theflowchart in FIG. 13, of the operation of the HDMI(R) source 53 in FIG.2.

The HDMI(R) source 53 waits for E-EDID of the HDMI(R) sink 61 to betransmitted from the HDMI(R) sink 61 via the DDC illustrated in FIG. 2,and receives the E-EDID in step S11.

Then, by referring to VSDB (FIG. 10) of E-EDID from the HDMI(R) sink 61,in step S12, the HDMI(R) source 53 recognizes which one of a 24-bitimage, a 30-bit image, a 36-bit image, and a 48-bit image the image (thecorresponding image) that can be received by the HDMI(R) sink 61 is.Furthermore, the HDMI(R) source 53 determines the image in the deepcolor mode, that is, the image to be transmitted through the three TMDSchannels #0 to #2, among the images supported by the HDMI(R) sink 61.

At this point, it is possible for the HDMI(R) source 53 to determine,for example, an image having the highest resolution as an image to betransmitted through the three TMDS channels #0 to #2 among the imagessupported by the HDMI(R) sink 61. In this case, when the HDMI(R) sink 61supports, for example, a 24-bit image, a 30-bit image, a 36-bit image,and a 48-bit image, the 48-bit image having the highest resolution isdetermined as an image to be transmitted through the TMDS channels #0 to#2.

Thereafter, in step S13, the HDMI(R) source 53 adjusts the frequency ofthe pixel clock, and thereby starts outputting the pixel clockcorresponding to the deep color mode determined in step S12. The processthen proceeds to step S14.

In step S14, the HDMI(R) source 53 starts, through the TMDS channels #0to #2, the transmission of the pixel data of the image indicated by thedeep color mode determined in step S12.

The transmission of the image indicated by the deep color mode throughthe TMDS channels #0 to #2 is performed in synchronization with thepixel clock whose output has been started in step S13.

Furthermore, during the transmission of the image indicated by the deepcolor mode through the TMDS channels #0 to #2, as illustrated in FIGS.11 and 12, the HDMI(R) source 53 transmits, for each video field, thatis, for each frame, a general control packet in which the bits CD0, CD1,and CD2 indicating the deep color mode of the image transmitted in thevideo data period are written, in the control period (FIG. 4) of thevertical blanking period.

Next, a description will be given, with reference to the flowchart inFIG. 14, of the operation of the HDMI(R) sink 61 of FIG. 2.

In step S31, the HDMI(R) sink 61 transmits its own E-EDID to the HDMI(R)source 53 via the DDC (FIG. 2).

Thereafter, as illustrated in FIG. 13, in the HDMI(R) source 53, theoutput of a pixel clock is started and a general control packet istransmitted via the TMDS channels #0 to #2. Then, in step S32, theHDMI(R) sink 61 receives the general control packet (FIGS. 11 and 12)from the HDMI(R) source 53, and recognizes the deep color mode of theimage transmitted in the video data period by referring to the bits CD0,CD1, and CD2 of the general control packet.

Then, the HDMI(R) sink 61 waits for the pixel data of the image of thedeep color mode, which was recognized in step S32, to be transmittedfrom the HDMI(R) source 53 in synchronization with the pixel clock viathe TMDS channels #0 to #2, and receives the pixel data in step S33.

The processing of steps S32 and S33 is performed for each video field.

Next, as illustrated in FIG. 13, the HDMI(R) source 53 determines thedeep color mode and transmits the image of the deep color mode throughthe TMDS channels #0 to #2. The image supplied as an object to betransmitted to the HDMI(R) source 53, that is, for example, the image tobe transmitted from the codec 52 (FIG. 1) to the HDMI(R) source 53, doesnot necessarily match the image of the deep color mode, which wasdetermined by the HDMI(R) source 53.

That is, there is a case in which the image to be transmitted is animage having a resolution lower than that in the deep color mode, whichwas determined by the HDMI(R) source 53.

At this point, the image to be transmitted from the HDMI(R) source 53 tothe HDMI(R) sink 61 will be hereinafter referred to as a main image, asappropriate.

In the manner described above, when the main image is an image having aresolution lower than that in the deep color mode, which was determinedby the HDMI(R) source 53, the pixel data of the main image istransmitted as shown in, for example, FIG. 15.

FIG. 15 shows transmission pixel data that is pixel data transmittedthrough one TMDS channel when the image in the deep color mode is a36-bit image and the main image is a 30-bit image having a resolutionlower than that of an image in the deep color mode.

When the image of the deep color mode is a 36-bit image, each of R, G,and B components of the pixel data of the 36-bit image is 12 bits.Therefore, as shown in FIG. 15, transmission pixel data transmittedthrough one TMDS channel is 12-bit data.

On the other hand, when the main image is a 30-bit image, each of R, G,and B components of the pixel data of the 30-bit image is 10 bits.Therefore, when transmitting a 30-bit image that is a main image, thepixel data of the main image to be transmitted through one TMDS channelis data of 10 bits, which is less than 12 bits, which is a number ofbits of the transmission pixel data.

In the manner described above, when the pixel data of the main image isan image with a number of bits smaller than that of transmission pixeldata, the HDMI(R) source 53 transmits the pixel data of the main imagein such a manner that, for example, as shown in FIG. 15, the pixel datais assigned to, by being packed closer to, the higher-order bits of thetransmission pixel data.

Therefore, when the pixel data of the main image is 10 bits and thetransmission pixel data is 12 bits, as shown in FIG. 15, the transmitter72 (FIG. 2) of the HDMI(R) source 53 assigns the pixel data of the10-bit main image to the high-order 10 bits of a 12-bit transmissionpixel data and transmits it.

In this case, the receiver 81 (FIG. 2) of the HDMI(R) sink 61 receivesthe 12-bit transmission pixel data from the transmitter 72 of theHDMI(R) source 53. Only the pixel data of the main image, which isassigned to the high-order 10 bits among the 12-bit transmission pixeldata, is processed, and the remaining low-order 2 bits are ignored(discarded).

At this point, when the HDMI(R) source 53 and the HDMI(R) sink 61support, for example, a 36-bit image in the manner described above, thatis, when the HDMI(R) source 53 transmits 12-bit transmission pixel dataand the HDMI(R) sink 61 can receive 12-bit transmission pixel data, inthe case that the pixel data of the main image is pixel data of lessthan 12 bits, in the HDMI(R) source 53 supporting a 36-bit image, thelow-order bits to which the pixel data of the main image is not assignedamong the 12-bit transmission pixel data are set as a no-signal (0).Then, in the HDMI(R) sink 61 supporting a 36-bit image, the 12-bittransmission pixel data from the HDMI(R) source 53 is processed as itis, and the thereby obtained image is displayed. In the display of theimage, since the low-order bits to which the pixel data of the mainimage is not assigned among the 12-bit transmission pixel data are ano-signal (0), the low-order bits are ignored in the display of theimage in the HDMI(R) sink 61. As a result, in the HDMI(R) sink 61, theimage formed by the pixel data assigned to the high-order bits among the12-bit transmission pixel data, that is, the main image, is displayed.Therefore, in the HDMI(R) sink 61, if 8-bit or 10-bit pixel data isassigned to the 12-bit transmission pixel data, an image formed by the8-bit or 10-bit pixel data is displayed.

In the manner described above, when a number B1 of bits of the pixeldata of the main image is smaller than a number B2 of bits of thetransmission pixel data, the pixel data of the main image whose numberof bits is B1 is assigned to the high-order bits of the transmissionpixel data whose number of bits is B2, and the transmission pixel datais transmitted from the HDMI(R) source 53 to the HDMI(R) sink 61. Inthis case, since the low-order (B2−B1) bits to which the pixel data ofthe main image is not assigned among the transmission pixel datatransmitted from the HDMI(R) source 53 to the HDMI(R) sink 61 are notsubstantially used, this is inefficient.

Therefore, it is possible for the HDMI(R) source 53 and the HDMI(R) sink61 to assign a signal differing from the main image (hereinafterreferred to as a “sub-signal” as appropriate) to bits to which the pixeldata of the main image is not assigned among the transmission pixeldata, thereby making it possible to perform efficient data transmissionin which the main image and the sub-signal are transmittedsimultaneously.

At this point, bits to which the pixel data of the main image is notassigned among the transmission pixel data will be hereinafter referredto remainder bits as appropriate.

FIG. 16 shows an assignment method of assigning a sub-signal totransmission pixel data.

For example, as illustrated in FIG. 15, when the transmission pixel data(pixel data transmitted through one TMDS channel) is 12 bits and (onecomponent of) the pixel data of the main image is 10 bits, the pixeldata of the 10-bit main image is assigned to the high-order 10 bits ofthe transmission pixel data, and thereby the low-order 2 bits of thetransmission pixel data become remainder bits.

If it is assumed that the sub-signal is a signal (data) in which, forexample, 8 bits is one unit, as shown in FIG. 16, the sub-signal in oneunit is divided into four pieces of data at intervals of two bits equalto the remainder bits, and the four pieces of the data are assigned tothe low-order 2 bits, which are the remainder bits of the transmissionpixel data of 4 pixels in a corresponding manner.

In this case, if the sub-signal is assumed to be an image in which, forexample, each of R, G, and B components is 8 bits, as this image that isthe sub-signal, for example, an image having a resolution of (the numberof pixels) approximately ¼ that of the main image can be used.

At this point, when the main image is an image of 1920 pixels× . . .1080 lines in which, for example, each of R, G, and B components is 10bits, and the transmission pixel data is, for example, 12 bits, in thecase that the sub-signal is an image of 480 pixels× . . . 360 lines inwhich each of R, G, and B components is 8 bits, the sub-signal as theimage of 480 pixels× . . . 360 lines can be assigned to the low-order 2bits that are the remainder bits of the transmission pixel data for 360lines among the 1080 lines of the main image. Then, in this case,another sub-signal can further be assigned to the transmission pixeldata for 720 (=1080−360) lines, to which the sub-signal as the image for480 pixels× . . . 360 lines is not assigned, among the transmissionpixel data for 1080 lines.

Furthermore, when the main image is an image of 1920 pixels× . . . 1080lines in which, for example, each of R, G, and B components is 8 bits,and the transmission pixel data is, for example, 12 bits, the low-order4 bits of the transmission pixel data become remainder bits. In thiscase, an image of approximately 960 pixels× . . . 720 lines in whicheach of R, G, and B components is 8 bits can be assigned to thetransmission pixel data for 1080 lines of the main image.

In the following, the sub-signal is assumed to be a signal in units of 8bits.

In the manner described above, HDMI(R) capable of assigning a sub-signalto the remainder bits of the transmission pixel data and transmittingtransmission pixel data together with the main image through the TMDSchannels #0 to #2 will be referred to as extended HDMI(R) in order tomake a distinction from the current HDMI(R). Then, when the HDMI(R)source in compliance with extended HDMI(R) assigns the sub-signal to theremainder bits of the transmission pixel data, first, it is necessary torecognize whether or not the HDMI(R) sink with which communication isperformed complies with extended HDMI(R).

Whether or not the HDMI(R) sink complies with extended HDMI(R) can bedescribed in, for example, E-EDID, which defines capability informationon the capability of the HDMI(R) sink, similarly to that describedabove, that is, whether or not the HDMI(R) sink complies with deep colorHDMI(R).

FIG. 17 shows the format of VSDB in E-EDID.

In the current HDMI(R), as shown in FIG. 10, the bits #4, #5, #6, and #7of the fifth, sixth, seventh, and eighth bits from the LSB of byte #7 ofVSDB are unused (Reserved). In FIG. 17, the bits Sub_(—)2 bit, Sub_(—)4bit, Sub_(—)8 bit, and Sub_Data_Support are assigned to the bits #4, #5,#6, and #7, respectively.

All of the bit Sub_(—)2 bit assigned to bit #4 of byte #7 of VSDB, thebit Sub_(—)4 bit assigned to bit #5, and the bit Sub_(—)8 bit assignedto bit #6 are set to 0 when the HDMI(R) sink cannot receive asub-signal, that is, when the HDMI(R) sink cannot handle a sub-signal.

Then, when the HDMI(R) sink can set the low-order 2 bits of thetransmission pixel data as remainder bits and can handle the sub-signalassigned to the 2-bit remainder bits, the bit Sub_(—)2 bit is set to 1.When the HDMI(R) sink can set the low-order 4 bits of the transmissionpixel data as remainder bits and can handle the sub-signal assigned tothe 4-bit remainder bits, the bit Sub_(—)4 bit is set to 1. Furthermore,when the HDMI(R) sink can set the low-order 8 bits of the transmissionpixel data as remainder bits and can handle the sub-signal assigned tothe 8-bit remainder bits, the bit Sub_(—)8 bit is set to 1.

The bit Sub_Data_Support is set to 1 when the HDMI(R) sink can handle asub-signal and is set to 0 when the HDMI(R) sink does not comply withextended HDMI(R).

When the bit Sub_Data_Support is 0, all of Sub_(—)2 bit, Sub_(—)4 bit,and Sub_(—)8 bit are set to 0.

In the manner described above, as a result of describing whether or notthe HDMI(R) sink complies with extended HDMI(R) in VSDB of E-EDID, it ispossible for the HDMI(R) source to recognize whether or not the HDMI(R)sink can handle the sub-signal by reading E-EDID from the HDMI(R) sinkand by referring to the VSDB of the E-EDID. Furthermore, when theHDMI(R) sink can handle the sub-signal, it is possible to recognize howmany of the low-order bits of the transmission pixel data can beassigned as remainder bits to the sub-signal.

The bits Sub_(—)2 bit, Sub_(—)4 bit, Sub_(—)8 bit, and Sub_Data_Supportshown in FIG. 17 can also be described in VSDB of E-EDID of the HDMI(R)source.

Although it is assumed in FIG. 17 that the sub-signal to be assigned tothe transmission pixel data is 2 bits, 4 bits, or 8 bits, the number ofbits of the sub-signal to be assigned to the transmission pixel data isnot limited to this.

Furthermore, it is possible to set a value corresponding to the numberof bits of the sub-signal, which is assigned to the transmission pixeldata, rather than Sub_(—)2 bit, Sub_(—)4 bit, Sub_(—)8 bit, andSub_Data_Support, in the 4 bits #4 to #7 of the fifth to eighth from theLSB of byte #7 of VSDB. In this case, 4 bits, that is, bits #4 to #7,can represent 16 kinds of number of bits.

Next, as described above, transmission and reception of E-EDID to andfrom the HDMI(R) source and the HDMI(R) sink is performed at a specifictiming, such as when the HDMI(R) source and the HDMI(R) sink areconnected to each other or when the power supply of the HDMI(R) sourceor the HDMI(R) sink is turned on, and is not performed in a periodicmanner.

On the other hand, when the HDMI(R) source and the HDMI(R) sink complywith extended HDMI(R), there can be a case in which a sub-signal hasbeen assigned to the transmission pixel data to be transmitted from theHDMI(R) source to the HDMI(R) sink and a case in which a sub-signal hasnot been assigned. Furthermore, when the sub-signal has been assigned tothe transmission pixel data, there is a case in which the assignedsub-signal is 2 bits, 4 bits, or 8 bits.

At this point, for the sake of simplicity of description, the fact thata sub-signal has not been assigned to transmission pixel data will alsobe hereinafter represented that the number of bits of the sub-signalassigned to the transmission pixel data is 0, as appropriate.

As illustrated in FIG. 4, the transmission of the transmission pixeldata is performed in the video data period assigned to the active videoperiod (Active Video) of the video field. Therefore, it is preferablethat the HDMI(R) sink can recognize, for each video field containing thevideo data period, which one of 0 bits, 2 bits, 4 bits, and 8 bits thesub-signal assigned to transmission pixel data transmitted in the videodata period is.

In this case, it is necessary to transmit, for each video field from theHDMI(R) source to the HDMI(R) sink, information (hereinafter referred toas “sub-signal information” as appropriate) indicating which one of 0bits, 2 bits, 4 bits, or 8 bits the sub-signal assigned to thetransmission pixel data transmitted in the video data period containedin the video field is.

Similarly to the above-described deep color mode, the sub-signalinformation can be contained in the general control packet transmittedin the control period (FIG. 4) in the vertical blanking period, and canbe transmitted from the HDMI(R) source to the HDMI(R) sink for eachvideo field.

FIG. 18 shows a general control packet containing sub-signalinformation.

As illustrated in FIG. 11, the general control packet has a packetheader (General Control Packet Header) and a subpacket (General ControlSubpacket). The upper area of FIG. 18 shows a packet header, and thelower area of FIG. 18 shows a subpacket.

In the current HDMI(R), it is assumed that the bits #0, #1, and #2 ofthe first, second, and third bits from the LSB of byte #SB2 of asubpacket of a general control packet are unused and set to 0. In FIG.18, bits SD0, SD1, and SD2 as sub-signal information are assigned tobits #0, #1, and #2, respectively.

FIG. 19 shows the relationship between the bits SD0, SD1, and SD2 ofbyte #SB2 of a subpacket, and the number of bits of a sub-signalassigned to transmission pixel data transmitted in the video data period(FIG. 4) contained in the video field containing the control period(FIG. 4) in which a general control packet having the subpacket istransmitted.

When the number of bits of the sub-signal assigned to the transmissionpixel data is 0, that is, when a sub-signal is not assigned to thetransmission pixel data (Sub Data not Inserted), all the bits SD0, SD1,and SD2 as sub-signal information are set to 0 in the same manner as inthe current HDMI (R).

Furthermore, when the number of bits assigned to the transmission pixeldata is 2, the bits SD0, SD1, and SD2 as the sub-signal information areset to, for example, 1, 0, and 0, respectively. When the number of bitsassigned to transmission pixel data is 4, the bits SD0, SD1, and SD2 asthe sub-signal information are set to, for example, 0, 1, and 0,respectively. When the number of bits assigned to transmission pixeldata is 8, the bits SD0, SD1, and SD2 as the sub-signal information areset to, for example, 1, 1, and 0, respectively.

In the manner described above, the HDMI(R) source transmits the bitsSD0, SD1, and SD2 as the sub-signal information, with the bits SD0, SD1,and SD2 being contained in the general control packet, in the controlperiod of the video field. As a result, it is possible for the HDMI(R)sink to recognize which one of 0 bits, 2 bits, 4 bits, and 8 bits thesub-signal assigned to the transmission pixel data transmitted in thevideo data period of the video field is.

FIG. 20 shows an example of the configuration of a source signalprocessor 71 possessed by the HDMI(R) source 53 when the HDMI(R) source53 of FIG. 2 complies with extended HDMI.

In FIG. 20, the source signal processor 71 includes a main imageprocessor 101, a sub-signal addition section 102, a sub-signal processor103, a sub-signal-related information insertion section 104, asub-signal reception capability/incapability determination section 105,a number of sub-signal assignment bits determination section 106, asub-signal frame information transmission controller 107, and adeep-color-mode determination section 108.

For example, a main image having each of R, G, and B components issupplied to the main image processor 101. The main image processor 101performs necessary processing on the main image supplied thereto, andsupplies the pixel data of the main image to the sub-signal additionsection 102.

Furthermore, the main image processor 101 detects the number P of pixels(the number of valid pixels) in the video data period of the video field(FIG. 4) of the main image supplied thereto, and supplies the number Pof pixels to the sub-signal processor 103.

Furthermore, the main image processor 101 detects the number B1 of bitsof each component of the pixel data of the main image supplied theretoand supplies the number B1 of bits to the number of sub-signalassignment bits determination section 106.

The pixel data of the main image whose number of bits is B1 is suppliedto the sub-signal addition section 102 from the main image processor101, and also a sub-signal is supplied to the sub-signal additionsection 102 from the sub-signal-related information insertion section104. Furthermore, the deep color mode is supplied to the sub-signaladdition section 102 from the deep-color-mode determination section 108and also, the number of sub-signal assignment bits indicating a numberB3 of bits of the sub-signal assigned to the transmission pixel data issupplied to the sub-signal addition section 102 from the number ofsub-signal assignment bits determination section 106.

The sub-signal addition section 102 recognizes the number B2 of bits ofthe transmission pixel data on the basis of the deep color mode suppliedfrom the deep-color-mode determination section 108. That is, forexample, when the deep color mode indicates one of a 24-bit image, a30-bit image, a 36-bit image, and a 48-bit image, the sub-signaladdition section 102 recognizes one of 8, 10, 12, and 16 as the numberB2 of bits of the transmission pixel data, respectively.

Then, the sub-signal addition section 102 divides the sub-signalsupplied from the sub-signal-related information insertion section 104for each number B3 of sub-signal assignment bits from the number ofsub-signal assignment bits determination section 106, and adds thedivision sub-signal, which is a divided sub-signal, as the low-orderbits of the pixel data whose number of bits is B1, which is suppliedfrom the main image processor 101. As a result, the transmission pixeldata having the number B2 of bits, which is recognized from the deepcolor mode from the deep-color-mode determination section 108, that is,the transmission pixel data having the number of bits B2 (=B1+B3), inwhich the pixel data of the main image whose number of bits is B1 isassigned to the high-order bits and the division sub-signal whose numberof bits is B3 is assigned to the low-order bits, is constructed.

The sub-signal addition section 102 constructs transmission pixel datawhose number of bits is B2 with regard to each of R, G, and Bcomponents. The transmission pixel data having each of R, G, and Bcomponents, whose number of bits is B2, which is obtained by thesub-signal addition section 102, is supplied to the transmitter 72 (FIG.2), and is transmitted, for example, at a timing corresponding to thenumber B2 of bits of the transmission pixel data among the timingsdescribed in FIGS. 6 to 9 through the TMDS channels #0 to #2.

As described above, the number P of bits in the video data period of themain image is supplied to the sub-signal processor 103 from the mainimage processor 101 and also, the sub-signal is supplied to thesub-signal processor 103. Furthermore, the number of sub-signalassignment bits is supplied to the sub-signal processor 103 from thenumber of sub-signal assignment bits determination section 106.

The sub-signal processor 103 determines the maximum amount of data P . .. × . . . B3 of the sub-signal that can be transmitted in the video dataperiod of one video field on the basis of the number P of bits in thevideo data period of the main image, which is supplied from the mainimage processor 101, and the number B3 of bits of the sub-signalassignment bits B3, which is supplied from the number of sub-signalassignment bits determination section 106, and determines the amount Dof data of the sub-signal (hereinafter referred to as an “amount ofsub-signal addition unit data” as appropriate) that is transmitted inthe video data period of one video field (1 frame) in the range of themaximum amount of data P . . . × . . . B3.

Furthermore, the sub-signal processor 103 supplies, for each of theamount D of the sub-signal addition unit data, the sub-signal suppliedthereto, to the sub-signal-related information insertion section 104 foreach of the amount D of the sub-signal addition unit data.

Furthermore, the sub-signal processor 103 supplies the sub-signalinformation indicating whether or not a sub-signal is contained in thetransmission pixel data to the sub-signal frame information transmissioncontroller 107.

That is, when the sub-signal of the amount D of the sub-signal additionunit data is to be supplied to the sub-signal-related informationinsertion section 104, that is, when there is a sub-signal to beassigned to the transmission pixel data, the sub-signal processor 103supplies the sub-signal information indicating that a sub-signal iscontained in the transmission pixel data to the sub-signal frameinformation transmission controller 107. Furthermore, when thesub-signal of the amount D of the sub-signal addition unit data is notto be supplied to the sub-signal-related information insertion section104, that is, when there is no sub-signal to be supplied to thesub-signal-related information insertion section 104, the sub-signalprocessor 103 supplies the sub-signal information indicating that asub-signal is not contained in the transmission pixel data to thesub-signal frame information transmission controller 107.

The sub-signal-related information insertion section 104 contains(inserts) sub-signal-related information related to the sub-signal, tothe sub-signal of the amount D of the sub-signal addition unit data fromthe sub-signal processor 103 and supplies the signal to the sub-signaladdition section 102.

The VSDB (FIG. 17) of the E-EDID, which is read from the HDMI(R) sinkwith which the HDMI(R) source 53 communicates, is supplied to thesub-signal reception capability/incapability determination section 105.

The sub-signal reception capability/incapability determination section105 determines whether or not the HDMI(R) sink with which the HDMI(R)source 53 communicates can receive the sub-signal, that is, determineswhether or not the HDMI(R) sink can handle the sub-signal, by referringto the bit Sub_Data_Support (FIG. 17) of the VSDB supplied thereto, andsupplies the determination result to necessary blocks.

Furthermore, when it is determined that the HDMI(R) sink with which theHDMI(R) source 53 communicates can handle the sub-signal, the sub-signalreception capability/incapability determination section 105 furtherrecognizes the number of bits of the sub-signal (hereinafter referred toas the “number of bits that can be handled” as appropriate) that can behandled by the HDMI(R) sink with which the HDMI(R) source 53communicates by referring to the bits Sub_(—)2 bit, Sub_(—)4 bit, andSub_(—)8 bit (FIG. 17) of the VSDB, and supplies the number of bits ofthe sub-signal to the number of sub-signal assignment bits determinationsection 106.

As described above, the number B1 of bits of the pixel data of the mainimage is supplied to the number of sub-signal assignment bitsdetermination section 106 from the main image processor 101. Also, thenumber of bits that can be handled is supplied to the number ofsub-signal assignment bits determination section 106 from the sub-signalreception capability/incapability determination section 105.Furthermore, the deep color mode is supplied to the number of sub-signalassignment bits determination section 106 from the deep-color-modedetermination section 108.

On the basis of the deep color mode supplied from the deep-color-modedetermination section 108, the number of sub-signal assignment bitsdetermination section 106 recognizes the number B2 of bits of thetransmission pixel data, and determines the difference B2−B1 with thenumber B1 of bits of the pixel data of the main image from the mainimage processor 101, that is, the number (B2−B1) of bits of theremainder bits of the transmission pixel data.

Then, when there is a number of bits matching the number (B2−B1) of bitsof the remainder bits of the transmission pixel data in the numbers ofbits that can be handled from the sub-signal receptioncapability/incapability determination section 105, the number ofsub-signal assignment bits determination section 106 determines thenumber of bits to be the number B3 of sub-signal assignment bits.

At this point, when the number of bits that can be handled from thesub-signal reception capability/incapability determination section 105is, for example, of three types of 2 bits, 4 bits, and 8 bits, thenumber B3 of sub-signal assignment bits is determined to be thefollowing values.

That is, when the number B2 of bits of the transmission pixel data is,for example, 10 and the number B1 of bits of the pixel data of the mainimage is, for example, 8, the number B3 of sub-signal assignment bits isdetermined to be 2.

Furthermore, when the number B2 of bits of the transmission pixel datais, for example, 12 and the number B1 of bits of the pixel data of themain image is, for example, 8 or 10, the number B3 of sub-signalassignment bits is determined to be 4 or 2.

When the number B2 of bits of the transmission pixel data is, forexample, 16 and the number B1 of bits of the pixel data of the mainimage is 8 or 12, the number B3 of sub-signal assignment bits isdetermined to be 8 or 4.

When there is no number of bits matching the number (B2−B1) of bits ofthe remainder bits of the transmission pixel data in the numbers of bitsthat can be handled from the sub-signal receptioncapability/incapability determination section 105, the numbers ofsub-signal assignment bits determination section 106 can determine, forexample, a maximum value among the number of bits that can be handledmatching the number of bits less than the number (B2−B1) of bits of theremainder bits of the transmission pixel data among the numbers of bitsthat can be handled from the sub-signal receptioncapability/incapability determination section 105, to be the number B3of sub-signal assignment bits. However, for the sake of simplicity ofdescription, when there is no number of bits matching the number (B2−B1)of bits of the remainder bits of the transmission pixel data in thenumbers of bits that can be handled from the sub-signal receptioncapability/incapability determination section 105, it is assumed thatthe HDMI(R) source 53 does not transmit the sub-signal and the HDMI(R)sink with which communication is performed performs the same processingas that when the sub-signal cannot be handled.

When the number B3 of sub-signal assignment bits is determined, thenumber of sub-signal assignment bits determination section 106 suppliesthe number B3 of sub-signal assignment bits to the sub-signal additionsection 102, the sub-signal processor 103, and the sub-signal frameinformation transmission controller 107.

As described above, sub-signal information indicating whether or not thesub-signal is contained in the transmission pixel data is supplied tothe sub-signal frame information transmission controller 107 from thesub-signal processor 103. Also, the number B3 of sub-signal assignmentbits is supplied to the sub-signal-frame information transmissioncontroller 107 from the number of sub-signal assignment bitsdetermination section 106. In addition, the deep color mode is suppliedto the sub-signal frame information transmission controller 107 from thedeep-color-mode determination section 108.

The sub-signal frame information transmission controller 107 allows thetransmitter 72 (FIG. 2) to transmit the sub-signal information from thesub-signal processor 103, the deep color mode from the deep-color-modedetermination section 108, and a general control packet (FIG. 18)containing the number B3 of sub-signal assignment bits, which issupplied from the number of sub-signal assignment bits determinationsection 106 as necessary.

That is, when the sub-signal information from the sub-signal processor103 indicates that the sub-signal is not contained in the transmissionpixel data, the sub-signal frame information transmission controller 107performs transmission control such that the transmitter 72 transmits ageneral control packet in which all the bits SD0, SD1, and SD2 of FIG.18 are set to 0, and the bits CD0, CD1, and CD2 are set to valuesindicating the deep color mode from the deep-color-mode determinationsection 108 (hereinafter referred to as a “general control packet withno sub-signal” as appropriate).

Furthermore, when the sub-signal information from the sub-signalprocessor 103 indicates that a sub-signal is contained in thetransmission pixel data, the sub-signal frame information transmissioncontroller 107 performs transmission control such that the transmitter72 transmits a general control packet (hereinafter referred to as a“general control packet with a sub-signal” as appropriate) in which thebits SD0, SD1, and SD2 of FIG. 18 are set to values indicating thenumber B3 of sub-signal assignment bits from the number of sub-signalassignment bits determination section 106 and the bits CD0, CD1, and CD2are set to values indicating the deep color mode from thedeep-color-mode determination section 108.

The VSDB (FIG. 17) of the E-EDID read from the HDMI(R) sink with whichthe HDMI(R) source 53 communicates is supplied to the deep-color-modedetermination section 108.

The deep-color-mode determination section 108 determines whether or notthe HDMI(R) sink with which the HDMI(R) source 53 communicates supportsa high resolution image by referring to the bits Suport_(—)30 bit,Suport_(—)36 bit, and Suport_(—)48 bit (FIG. 17) of the VSDB suppliedthereto. When it is determined that the HDMI(R) sink does not support ahigh resolution image, the deep-color-mode determination section 108determines the image in the deep color mode, that is, the image to betransmitted through three TMDS channels #0 to #2, to be a 24-bit image.

Furthermore, when it is determined that the HDMI(R) sink with which theHDMI(R) source 53 communicates supports a high resolution image, thedeep-color-mode determination section 108 further refers to the bitsSuport_(—)30 bit, Suport_(—)36 bit, and Suport_(—)48 bit (FIG. 17) ofthe VSDB, thereby recognizes the high resolution image supported by theHDMI(R) sink with which the HDMI(R) source 53 communicates anddetermines the image in the deep color mode, that is, the image to betransmitted through the three TMDS channels #0 to #2 among the highresolution images supported by the HDMI(R) sink.

That is, the deep-color-mode determination section 108 determines, forexample, the image having the highest resolution among the imagessupported by, for example, the HDMI(R) sink, to be an image in the deepcolor mode (the image to be transmitted through the three TMDS channels#0 to #2).

Then, the deep-color-mode determination section 108 supplies the deepcolor mode to the sub-signal addition section 102, the number ofsub-signal assignment bits determination section 106, and the sub-signalframe information transmission controller 107.

Next, a description will be given, with reference to FIG. 21, ofsub-signal-related information inserted into a sub-signal by thesub-signal-related information insertion section 104 of FIG. 20.

In extended HDMI(R), a sub-signal is assigned to the pixel data in thevideo data period of a video field (FIG. 4), that is, to the low-orderbits of the transmission pixel data, and the sub-signal together withthe main image assigned to the high-order bits of the transmission pixeldata is transmitted. However, the sub-signal is not necessarily assignedto all the transmission pixel data.

That is, depending on the amount of data of the sub-signal, thesub-signal may be assigned to only the transmission pixel data in aportion of the video data period, and the sub-signal may not be assignedto the remaining transmission pixel data.

In the manner described above, when a sub-signal is assigned to thetransmission pixel data in a portion of the video data period and thesub-signal is not assigned to the remaining transmission pixel data, itis necessary for the HDMI(R) sink that receive such transmission pixeldata to distinguish between the transmission pixel data to which thesub-signal has been assigned and the transmission pixel data to whichthe sub-signal has not been assigned, and to extract, as the sub-signal,the low-order bits of only the transmission pixel data to which thesub-signal has been assigned.

Therefore, the sub-signal-related information insertion section 104inserts sub-signal-related information containing at least informationused to distinguish the transmission pixel data to which the sub-signalhas been assigned into the sub-signal of the amount D of the sub-signaladdition unit data, that is, the sub-signal assigned to the transmissionpixel data in the video data period of one video field (FIG. 4).

That is, the sub-signal-related information is composed of, for example,sub-signal start information and sub-signal end information, as shown inthe right side of FIG. 21. The sub-signal start information is locatedat the beginning of the sub-signal of the amount D of the sub-signaladdition unit data, and the sub-signal end information is located at theend of the sub-signal of the amount of the sub-signal addition unitdata.

Then, the sub-signal start information is assigned to the transmissionpixel data of the pixels constituting the first line (the firsthorizontal line from the top) in the video data period of the videofield, shown at the left of FIG. 21, and the sub-signal of the amount ofthe sub-signal addition unit data is sequentially assigned to thetransmission pixel data of the pixels constituting the second andsubsequent lines.

If it is assumed that all the sub-signals of the amount D of thesub-signal addition unit data are assigned to the transmission pixeldata of the pixels constituting the second to (M+1)th lines, thesub-signal end information is assigned to the transmission pixel data ofthe pixels constituting the (M+2)th line immediately after the (M+1)thline.

In the manner described above, in the sub-signal start informationlocated at the beginning of the sub-signal of the amount D of thesub-signal addition unit data, for example, the fact that a sub-signalhas been assigned to the transmission pixel data of the pixels of thesecond and subsequent lines, information indicating the type (attribute)of the sub-signal such that the sub-signal is image data, audio data, ortext data, the format of the sub-signal, and other information relatedto the sub-signal, can be contained.

In the sub-signal end information located at the end of the sub-signalof the amount of the sub-signal addition unit data, a unique codeindicating the end of the sub-signal can be contained. Furthermore, whenthe transmission pixel data to which the end of the sub-signal has beenassigned is transmission pixel data of the pixel in the middle of thepixels constituting the (M+1)th line, information indicating theposition of the pixel of the transmission pixel data can be contained inthe sub-signal end information.

When one line in the video data period of the video field is composedof, for example, 1920 pixels, in the case that the sub-signal isassigned to, for example, the low-order 2 bits of the transmission pixeldata, the amount of data that can be assigned to the transmission pixeldata of the pixels constituting one line is 2 bits× . . . 1920pixels=3840 bits=480 bytes. Therefore, as the sub-signal startinformation and the sub-signal end information, 480-byte information canbe used for each of them.

Furthermore, the sub-signal processor 103 of FIG. 20 determines theamount D of the sub-signal addition unit data in such a manner that theamount of data such that the amount of data of the sub-signal startinformation and the sub-signal end information contained in thesub-signal is added to the amount D of the sub-signal addition unit datadoes not exceed the maximum amount of data P . . . × . . . B3 of thesub-signal that can be transmitted in the video data period.

Next, a description will be given, with reference to the flowchart inFIG. 22, of the operation of the HDMI(R) source 53 when the HDMI(R)source 53 of FIG. 2 complies with extended HDMI(R) and the source signalprocessor 71 of the HDMI(R) source 53 is configured as shown in FIG. 20.

In the HDMI(R) source 53, the pixel data of the main image is suppliedto the main image processor 101 of the source signal processor 71 (FIG.20), and furthermore, a sub-signal is supplied to the sub-signalprocessor 103 as necessary.

The main image processor 101 performs necessary processing on the mainimage supplied thereto and supplies the pixel data of the processed mainimage to the sub-signal addition section 102.

Furthermore, the main image processor 101 detects the number P of pixels(the number of valid pixels) in the video data period of the video field(FIG. 4) of the main image supplied thereto, and supplies the number Pof pixels to the sub-signal processor 103.

Furthermore, the main image processor 101 detects the number B1 of bitsof each component of the pixel data of the main image, and supplies thenumber B1 of bits to the number of sub-signal assignment bitsdetermination section 106.

Furthermore, the HDMI(R) source 53 waits for the E-EDID of the HDMI(R)sink 61 to be transmitted via the DDC illustrated in FIG. 2 from theHDMI(R) sink 61, and receives the E-EDID in step S101.

In the HDMI(R) source 53, the E-EDID from the HDMI(R) sink 61 issupplied to the sub-signal reception capability/incapabilitydetermination section 105 of the source signal processor 71 (FIG. 20)and the deep-color-mode determination section 108.

In step S102, the deep-color-mode determination section 108 recognizeswhich one of a 24-bit image, a 30-bit image, a 36-bit image, and a48-bit image the image supported by the HDMI(R) sink 61 is by referringto the VSDB (FIG. 17) of the E-EDID from the HDMI(R) sink 61.Furthermore, the deep-color-mode determination section 108 determinesthe image to be transmitted through the three TMDS channels #0 to #2from among the images supported by the DMI(R) sink 61, and supplies thedeep color mode indicating the image to the sub-signal addition section102, the number of sub-signal assignment bits determination section 106,and the sub-signal frame information transmission controller 107.

When the image supported by the HDMI(R) sink 61 is only a 24-bit image,that is, when the HDMI(R) sink 61 does not support a high resolutionimage, the HDMI(R) source 53 does not perform processing of step S104and subsequent steps (to be described later) and performs processing incompliance with the current HDMI(R). Therefore, in this case, thetransmission of the sub-signal is not performed.

Thereafter, in step S103, the HDMI(R) source 53 adjusts the frequency ofthe pixel clock to a frequency corresponding to the deep color modedetermined in step S102, and starts outputting a pixel clock. Theprocess then proceeds to step S104.

In step S104, the sub-signal reception capability/incapabilitydetermination section 105 determines whether or not the HDMI(R) sink 61can receive a sub-signal, that is, can handle a sub-signal, by referringto the bit Sub_Data_Support (FIG. 17) of the VSDB of the E-EDID from theHDMI(R) sink 61, which is supplied thereto.

When it is determined in step S104 that the HDMI(R) sink 61 cannothandle a sub-signal, that is, when the bit Sub_Data_Support (FIG. 17) ofthe VSDB has been set to 0 indicating that the sub-signal cannot behandled, the process proceeds to step S105. Hereafter, the HDMI(R)source 53 does not transmit a sub-signal and transmits the pixel data ofthe image indicated by the deep color mode determined in step S102through the TMDS channels #0 to #2.

That is, in step S105, the sub-signal frame information transmissioncontroller 107 allows the transmitter 72 (FIG. 2) to transmit, in thecontrol period (FIG. 4) of the vertical blanking period of the videofield, a general control packet having no sub-signal, that is, a generalcontrol packet in which all the bits SD0, SD1, and SD2 in FIG. 18 areset to 0, and the bits CD0, CD1, and CD2 are set to values indicatingthe deep color mode supplied from the deep-color-mode determinationsection 108 in step S102. The process then proceeds to step S106.

In step S106, the sub-signal addition section 102 constructs thetransmission pixel data of the number B2 of bits corresponding to theimage of the deep color mode determined in step S102 on the basis ofonly the pixel data of the main image supplied from the main imageprocessor 101, and supplies the transmission pixel data to thetransmitter 72. As a result, the transmitter 72 transmits thetransmission pixel data in the video data period of the video field, andthe process then proceeds to step S107.

At this point, the transmission of the transmission pixel data isperformed in synchronization with the pixel clock whose output has beenstarted in step S103.

In step S107, the sub-signal addition section 102 determines whether ornot the transmission pixel data that has not yet been transmitted existsin the active video period of the video field (hereinafter referred toas a “video field of interest” as appropriate) in which the transmissionpixel data has been transmitted in the immediately previous step S106.

When it is determined in step S107 that the transmission pixel data thathas not yet been transmitted exists in the active video period of thevideo field of interest, the process returns to step S106, where thetransmission pixel data that has not yet been transmitted in the activevideo period of the video field of interest is transmitted.

When it is determined in step S107 that the transmission pixel data thathas not yet been transmitted does not exist in the active video periodof the video field of interest, that is, when the transmission of allthe transmission pixel data in the active video period of the videofield of interest is completed, the process proceeds to step S108, wherethe sub-signal addition section 102 determines whether or not a videofield (frame) next to the video field of interest exists.

When it is determined in step S108 that the video field (frame) next tothe video field of interest exists, the next video field is newly set asa video field of interest. The process then returns to step S105 andhereinafter, the same processing is repeated.

When it is determined in step S108 that the video field (frame) next tothe video field of interest does not exist, the processing is completed.

On the other hand, when it is determined in step S104 that the HDMI(R)sink 61 can handle a sub-signal, that is, when the bit Sub_Data_Support(FIG. 17) of the VSDB is 1, which is a value indicating that asub-signal can be handled, the sub-signal receptioncapability/incapability determination section 105 recognizes the numberof bits that can be handled, which is the number of bits of thesub-signal that can be handled by the HDMI(R) sink 61, by referring tothe bits Sub_(—)2 bit, Sub_(—)4 bit, and Sub_(—)8 bit (FIG. 17) of theVSDB, and supplies the number of bits to the number of sub-signalassignment bits determination section 106. The process then proceeds tostep S109.

In step S109, on the basis of the number B1 of bits of the pixel data ofthe main image from the main image processor 101, the number of bitsthat can be handled from the sub-signal receptioncapability/incapability determination section 105, and the number B2 ofbits of the transmission pixel data recognized from the deep color modefrom the deep-color-mode determination section 108, the number ofsub-signal assignment bits determination section 106 determines thenumber B3 of bits, which is the number of bits of the sub-signalassigned to the transmission pixel data in the manner described above,and supplies the number B3 of bits to the sub-signal addition section102, the sub-signal processor 103, and the sub-signal frame informationtransmission controller 107. The process then proceeds to step S110.

At this point, when the number P of bits in the video data period of themain image is supplied to the sub-signal processor 103 from the mainimage processor 101, and the number B3 of bits of sub-signal assignmentbits is supplied to the sub-signal processor 103 from the number ofsub-signal assignment bits determination section 106, as describedabove, the sub-signal processor 103 determines the amount D of thesub-signal addition unit data, which is the amount of data of thesub-signal to be transmitted in the video data period of one videofield, on the basis of the number P of bits in the video data period ofthe main image and the number B3 of sub-signal assignment bits.

Then, in step S110, the sub-signal processor 103 determines whether ornot there is a sub-signal to be added to the pixel data in the videodata period of the video field.

When it is determined in step S110 that there is a sub-signal to beadded to the pixel data in the video data period of the video field,that is, when, for example, a sub-signal has been supplied to thesub-signal processor 103, the sub-signal processor 103 supplies thesub-signal for only the amount D of the sub-signal addition unit datawithin the sub-signal supplied thereto to the sub-signal-relatedinformation insertion section 104 and also supplies sub-signalinformation indicating that a sub-signal has been contained in thetransmission pixel data to the sub-signal frame information transmissioncontroller 107. The process then proceeds to step S111, where theHDMI(R) source 53 transmits the sub-signal and the pixel data of themain image indicated by the deep color mode determined in step S102through the TMDS channels #0 to #2.

That is, in step S111, on the basis of the sub-signal information fromthe sub-signal processor 103, the sub-signal frame informationtransmission controller 107 allows the transmitter 72 (FIG. 2) totransmit, in the control period (FIG. 4) of the vertical blanking periodof the video field, a general control packet having a sub-signal, thatis, a general control packet in which the bits SD0, SD1, and SD2 in FIG.18 are set to values corresponding to the number B3 of sub-signalassignment bits, which is the number of bits of the sub-signal assignedto the transmission pixel data, which is supplied from the number ofsub-signal assignment bits determination section 106 in step S109, andthe bits CD0, CD1, and CD2 are set to values indicating the deep colormode supplied from the deep-color-mode determination section 108 in stepS102. The process then proceeds to step S112.

At this point, when it is determined in step S110 that there is asub-signal to be added to the pixel data in the video data period of thevideo field, as described above, the sub-signal processor 103 suppliesthe sub-signal for the amount D of the sub-signal addition unit datawithin the sub-signal supplied thereto to the sub-signal-relatedinformation insertion section 104.

When the sub-signal for the amount D of the sub-signal addition unitdata is supplied to the sub-signal-related information insertion section104 from the sub-signal processor 103, the sub-signal-relatedinformation insertion section 104 inserts sub-signal-related informationrelated to the sub-signal as illustrated in FIG. 21 and supplies it tothe sub-signal-addition section 102.

In step S112, the sub-signal addition section 102 starts theconstruction of the transmission image data, that is, the addition ofthe sub-signal from the sub-signal-related information insertion section104 to the pixel data of the main image from the main image processor101.

That is, the sub-signal addition section 102 divides the sub-signal fromthe sub-signal-related information insertion section 104 into divisionsub-signals at the intervals of the number B3 of sub-signal assignmentbits supplied from the number of sub-signal assignment bitsdetermination section 106 in step S109, and adds the division sub-signalas the low-order bits of the pixel data from the main image processor101. As a result, the transmission pixel data of the number B2 of bitsrecognized from the deep color mode supplied from the deep-color-modedetermination section 108 in step S102, that is, the transmission pixeldata whose number of bits is B2, in which the pixel data of the mainimage whose number of bits is B1 has been assigned to the high-orderbits and the division sub-signal whose number of bits is B3 has beenassigned to the low-order bits, is constructed.

Then, the sub-signal addition section 102 proceeds from step S112 tostep S113, where the sub-signal addition section 102 supplies thetransmission pixel data in which the pixel data of the main image hasbeen assigned to the high-order bits and the division sub-signal hasbeen assigned to the low-order bits to the transmitter 72. As a result,the transmitter 72 transmits the transmission pixel data in the videodata period of the video field, and the process then proceeds to stepS114.

At this point, the transmission of the transmission pixel data isperformed in synchronization with the pixel clock whose output has beenstarted in step S103.

In step S114, the sub-signal addition section 102 determines whether ornot the transmission pixel data that has not yet been transmitted existsin the active video period of the video field of interest, which is avideo field in which pixel data as transmission pixel data has beentransmitted in the immediately previous step S113.

When it is determined in step S114 that the transmission pixel data thathas not yet been transmitted exists in the active video period of thevideo field of interest, the process returns to step S113, where pixeldata that has not yet been transmitted in the active video period videofield of interest is transmitted as transmission pixel data.

Furthermore, when it is determined in step S114 that the transmissionpixel data that has not yet been transmitted does not exist in theactive video period of the video field of interest, that is, when thetransmission of all the transmission pixel data in the active videoperiod of the video field of interest is completed, the process proceedsto step S115, where the sub-signal addition section 102 determineswhether or not there is a video field next to the video field ofinterest.

When it is determined in step S115 that there is a video field next tothe video field of interest, the next video field is newly set as avideo field of interest. The process then returns to step S110 andhereinafter, the same processing is repeated.

When it is determined in step S115 that there is no video field next tothe video field of interest, the processing is completed.

On the other hand, when it is determined in step S110 that there is nosub-signal to be added to the pixel data in the video data period of thevideo field, that is, for example, when a sub-signal has not beensupplied to the sub-signal processor 103, the sub-signal processor 103supplies the sub-signal information indicating that a sub-signal is notcontained in the transmission pixel data to the sub-signal frameinformation transmission controller 107. The process then proceeds tostep S116, where, on the basis of the sub-signal information from thesub-signal processor 103, in the same manner as in step S105, thesub-signal frame information transmission controller 107 allows thetransmitter 72 (FIG. 2) to transmit, in the control period (FIG. 4) ofthe vertical blanking period of the video field, a general controlpacket having no sub-signal, that is, a general control packet in whichall the bits SD0, SD1, and SD2 in FIG. 18 are set to 0 and the bits CD0,CD1, and CD2 are set to values indicating the deep color mode suppliedfrom the deep-color-mode determination section 108 in step S102.

Then, hereinafter, in steps S113 to S115, the HDMI(R) source 53 does nottransmit a sub-signal and transmits the pixel data of the main imageindicated by the deep color mode determined in step S102 through theTMDS channels #0 to #2.

That is, when a general control packet having no sub-signal has beentransmitted in step S116, in step S113, the sub-signal addition section102 constructs transmission pixel data for the number B2 of bitscorresponding to the image of the deep color mode determined in stepS102 on the basis of only the pixel data of the main image supplied fromthe main image processor 101, and supplies the transmission pixel datato the transmitter 72. As a result, the transmitter 72 transmits thetransmission pixel data in the video data period of the video field, andthe process then proceeds to step S114.

At this point, the transmission of the transmission pixel data isperformed in synchronization with the pixel clock whose output has beenstarted in step S103.

In step S114, the sub-signal addition section 102 determines whether ornot transmission pixel data that has not yet been transmitted exists inthe active video period of the video field of interest, which is a videofield in which pixel data has been transmitted as transmission pixeldata in the immediately previous step S113.

When it is determined in step S114 that transmission pixel data that hasnot yet been transmitted exists in the active video period of the videofield of interest, the process returns to step S113, where thetransmission pixel data that has not yet been transmitted in the activevideo period of the video field of interest is transmitted.

When it is determined in step S114 that transmission pixel data that hasnot yet been transmitted does not exist in the active video period ofthe video field of interest, that is, when the transmission of all thetransmission pixel data in the active video period of the video field ofinterest has been completed, the process proceeds to step S115, wherethe sub-signal addition section 102 determines whether or not a videofield next to the video field of interest exists.

When it is determined in step S115 that a video field next to the videofield of interest exists, the next video field is newly set as a videofield of interest. The process then returns to step S110 andhereinafter, the same processing is repeated.

When it is determined in step S115 that a video field next to the videofield of interest does not exist, the processing is completed.

FIG. 23 shows an example of the configuration of a sink signal processor82 possessed by the HDMI(R) sink 61 when the HDMI(R) sink 61 of FIG. 2complies with extended HDMI.

In FIG. 23, the sink signal processor 82 includes an FIFO (First InFirst Out) memory 121, a sub-signal presence/absence determinationsection 122, a separator 123, a main image processor 124, a main imagememory 125, a sub-signal processor 126, and a sub-signal memory 127.

Transmission pixel data in the video data period of the video field(FIG. 4), which is received by the receiver 81 (FIG. 2), is supplied tothe FIFO memory 121.

The FIFO memory 121 sequentially stores the transmission pixel data fromthe receiver 81, and supplies it to the separator 123.

A general control packet in the control period of the vertical blankingperiod of the video field (FIG. 4), which was received by the receiver81, is supplied to the sub-signal presence/absence determination section122.

On the basis of the bits SD0, SD1, and SD2 of the general control packet(FIG. 18) from the receiver 81, the sub-signal presence/absencedetermination section 122 determines whether or not a sub-signal iscontained in the transmission pixel data transmitted in the video dataperiod immediately after the vertical blanking period in which thegeneral control packet has been transmitted, and supplies thedetermination result to the separator 123.

Furthermore, when it is determined that a sub-signal is contained in thetransmission pixel data, on the basis of the bits SD0, SD1, and SD2 ofthe general control packet (FIG. 18) from the receiver 81, thesub-signal presence/absence determination section 122 recognizes thenumber B3 of sub-signal assignment bits of the sub-signal contained inthe transmission pixel data transmitted in the video data periodimmediately after the vertical blanking period in which the generalcontrol packet has been transmitted, and supplies the number B3 ofsub-signal assignment bits of the sub-signal to the separator 123.

When the determination result indicating that a sub-signal is notcontained in the transmission pixel data is supplied from the sub-signalpresence/absence determination section 122, the separator 123 receivesthe transmission pixel data from the FIFO memory 121, and supplies thepixel data of the main image assigned to the transmission pixel data tothe main image processor 124.

Furthermore, when the determination result indicating that a sub-signalis contained in the transmission pixel data is supplied from thesub-signal presence/absence determination section 122, the separator 123receives the transmission pixel data from the FIFO memory 121, andseparates the pixel data of the main image and the division sub-signalfrom the transmission pixel data on the basis of the number B3 ofsub-signal assignment bits supplied from the sub-signal presence/absencedetermination section 122.

That is, the separator 123 extracts, as a division sub-signal, thelow-order bits for only the number B3 of sub-signal assignment bits fromthe sub-signal presence/absence determination section 122 among thetransmission pixel data from the FIFO memory 121, and supplies thedivision sub-signal to the sub-signal processor 126. Furthermore, theseparator 123 extracts, as the pixel data of the main image, theremaining high-order bits among the transmission pixel data from theFIFO memory 121, and supplies it to the main image processor 124.

The main image processor 124 performs necessary processing on the pixeldata of the main image supplied from the separator 123, reconstructs amain image for one video field, and supplies the main image to the mainimage memory 125.

The main image memory 125 temporarily stores the main image suppliedfrom the main image processor 124. The main image stored in the mainimage memory 125 is read as appropriate and is supplied to the displaycontroller 62 (FIG. 1).

The sub-signal processor 126 reconstructs the original sub-signal fromthe division sub-signal supplied from the separator 123 and supplies thesub-signal to the sub-signal memory 127. As illustrated in FIG. 21, thesub-signal-related information is contained in the sub-signal, and thesub-signal processor 126 reconstructs the sub-signal by referring to thesub-signal-related information as necessary.

The sub-signal memory 127 temporarily stores the sub-signal suppliedfrom the sub-signal processor 126.

Next, a description will be given, with reference to the flowchart inFIG. 24, of the operation of the HDMI(R) sink 61 when the HDMI(R) sink61 of FIG. 2 complies with extended HDMI(R) and the sink signalprocessor 82 of the HDMI(R) sink 61 is configured as shown in FIG. 23.

In step S131, the HDMI(R) sink 61 transmits its own E-EDID to theHDMI(R) sink 61 via the DDC (FIG. 2).

Thereafter, as illustrated in FIG. 22, in the HDMI(R) sink 61, theoutput of the pixel clock is started, and the general control packet istransmitted via the TMDS channels #0 to #2. Then, in step S132, thereceiver 81 (FIG. 2) of the HDMI(R) sink 61 receives the general controlpacket (FIG. 18) from the HDMI(R) sink 61, and supplies the generalcontrol packet to the sub-signal presence/absence determination section122 (FIG. 23) of the sink signal processor 82. The process then proceedsto step S133.

In step S133, on the basis of the bits SD0, SD1, and SD2 of the generalcontrol packet (FIG. 18) from the receiver 81, the sub-signalpresence/absence determination section 122 determines whether or not asub-signal is contained in the transmission pixel data transmitted inthe video data period immediately after the vertical blanking period inwhich the general control packet has been transmitted.

When it is determined in step S133 that a sub-signal is not contained inthe transmission pixel data, the sub-signal presence/absencedetermination section 122 supplies the determination result indicatingthe fact to the separator 123. The process then proceeds to step S134.

In step S134, the receiver 81 (FIG. 2) of the HDMI(R) sink 61 waits forthe transmission pixel data of the image of the deep color modeindicated by the bits CD0, CD1, and CD2 of the general control packetfrom the HDMI(R) source 53 to be transmitted from the HDMI(R) sink 61via the TMDS channels #0 to #2 in synchronization with the pixel clock,receives the transmission pixel data, and supplies it to the separator123 via the FIFO memory 121 of the sink signal processor 82. The processthen proceeds to step S135.

In step S135, on the basis of the determination result indicating that asub-signal is not contained in the transmission pixel data from thesub-signal presence/absence determination section 122, the separator 123supplies the pixel data of the main image assigned to the transmissionpixel data supplied via the FIFO memory 121 to the main image processor124.

Furthermore, in step S135, in order to reconstruct a main image for onevideo field, the main image processor 124 supplies the pixel data of themain image from the separator 123 to the main image memory 125, wherebyit is stored. The process then proceeds to step S136.

In step S136, the main image processor 124 determines whether or not theprocessing of the pixel data of the main image for one video field hasbeen completed, that is, the main image for one video field has beenstored in the main image memory 125.

When it is determined in step S136 that the processing of the pixel dataof the main image for one video field has not been completed, theprocess returns to step S134 and hereinafter, the same processing isrepeated.

Furthermore, when it is determined in step S136 that the processing ofthe pixel data of the main image for one video field has been completed,waiting for a general control packet to be transmitted in the next videofield is performed. The process then returns to step S132 andhereinafter, the same processing is repeated.

On the other hand, when it is determined in step S133 that a sub-signalis contained in the transmission pixel data, on the basis of the bitsSD0, SD1, and SD2 of the general control packet (FIG. 18) from thereceiver 81, the sub-signal presence/absence determination section 122recognizes the number B3 of sub-signal assignment bits of the sub-signalcontained in the transmission pixel data, and supplies it together withthe determination result indicating that a sub-signal has been containedin the transmission pixel data to the separator 123. The process thenproceeds to step S137.

In step S137, the receiver 81 (FIG. 2) of the HDMI(R) sink 61 waits forthe transmission pixel data of the deep color mode indicated by the bitsCD0, CD1, and CD2 of the general control packet from the HDMI(R) source53 to be transmitted from the HDMI(R) sink 61 via the TMDS channels #0to #2 in synchronization with the pixel clock, receives the transmissionpixel data, and supplies it to the separator 123 via the FIFO memory 121of the sink signal processor 82. The process then proceeds to step S138.

In step S138, on the basis of the determination result indicating that asub-signal is contained in the transmission pixel data from thesub-signal presence/absence determination section 122, the separator 123separates, from the transmission pixel data supplied via the FIFO memory121, the low-order bits for the number B3 of sub-signal assignment bitsfrom the sub-signal presence/absence determination section 122, andsupplies the low-order bits as a division sub-signal to the sub-signalprocessor 126.

Furthermore, in step S138, the separator 123 separates the remaininghigh-order bits from the transmission pixel data supplied via the FIFOmemory 121, and supplies the remaining high-order bits as the pixel dataof the main image to the main image processor 124. The process thenproceeds to step S139.

In step S139, in order to reconstruct a main image for one video field,the main image processor 124 supplies the pixel data of the main imagefrom the separator 123 to the main image memory 125, whereby it isstored. Furthermore, in step S139, in order to reconstruct a sub-signal,the sub-signal processor 126 supplies the division sub-signal from theseparator 123 to the sub-signal memory 127, whereby it is stored.

In step S140, the main image processor 124 determines whether or not theprocessing of the pixel data of the main image for one video field hasbeen completed, that is, a main image for one video field has beenstored in the main image memory 125.

When it is determined in step S140 that the processing of the pixel dataof the main image for one video field has not been completed, theprocess returns to step S137 and hereinafter, the same processing isrepeated.

Furthermore, when it is determined in step S140 that the processing ofthe pixel data of the main image for one video field has been completed,waiting for a general control packet to be transmitted in the next videofield is performed. The process then returns to step S132 andhereinafter, the same processing is repeated.

In the manner described above, in the HDMI(R) source 53 that receivesE-EDID as capability information indicating the capability of theHDMI(R) sink 61 (FIG. 2) and thereafter transmits the pixel data of anuncompressed image for one screen in one direction to the HDMI(R) sink61 by using a differential signal through the three TMDS channels #0 to#2 for transmitting a fixed number of bits of data per clock of a pixelclock in the video data period assigned to the active video period (thevalid image period) in which the horizontal blanking period and thevertical blanking period are excluded from the video field (the periodfrom one vertical synchronization signal to the next verticalsynchronization signal), the transmitter 72 transmits the transmissionpixel data to which a number of bits greater than 8 bits, which is afixed number of bits, has been assigned, in one direction to the HDMI(R)sink 61 by using a differential signal through the three TMDS channels#0 to #2 by adjusting the frequency of the pixel clock.

In this case, the HDMI(R) source 53 determines whether or not theHDMI(R) sink 61 can receive a sub-signal on the basis of the VSDB (FIG.17) of the E-EDID. When the HDMI(R) sink 61 can receive the sub-signal,the transmission pixel data is constructed by adding the sub-signal tothe pixel data of the main image composed of pixel data having a numberof bits less than that of the transmission pixel data transmitted by thetransmitter 72, and is transmitted through the three TMDS channels #0 to#2 by the transmitter 72.

Furthermore, in the HDMI(R) source 53, a general control packet (FIG.18) containing the bits SD0, SD1, and SD2 serving as sub-signalinformation indicating whether or not a sub-signal is contained in thetransmission pixel data transmitted in the video data period immediatelyafter the vertical blanking period is transmitted in the control period(FIG. 4) of the vertical blanking period.

On the other hand, in the HDMI(R) sink 61 that transmits E-EDID andthereafter receives pixel data transmitted using a differential signalthrough the three TMDS channels #0 to #2 from the HDMI(R) source 53, thereceiver 81 receives the transmission pixel data transmitted using adifferential signal through the three TMDS channels #0 to #2.

Furthermore, in the HDMI(R) sink 61, on the basis of the bits SD0, SD1,and SD2 contained in the general control packet (FIG. 18) transmitted inthe control period (FIG. 4) of the vertical blanking period, it isdetermined whether or not a sub-signal has been contained in thetransmission pixel data transmitted in the video data period immediatelyafter the vertical blanking period. When a sub-signal is contained inthe transmission pixel data, the sub-signal is separated from thetransmission pixel data.

Therefore, when the number of bits of the pixel data of the main imageis smaller than the number of bits of the transmission pixel datadetermined by the deep color mode, efficient data transmission such thatthe sub-signal is assigned to the bits of the transmission pixel data,which are not assigned to the pixel data of the main image, and thesub-signal is transmitted together with the main image can be performed.

The greater the difference between the number of bits of thetransmission pixel data and the number of bits of the pixel data of themain image, a sub-signal (division sub-signal) having a larger amount ofdata can be assigned to the transmission pixel data.

In this embodiment, no particular mention has been made in the uses ofthe sub-signal, but the sub-signal can be used for various kinds ofuses.

More specifically, as the sub-signal, for example, an image having a lowresolution synchronized with a main image, an image of a programdiffering from a main image, and other images can be used. In this case,it is possible for the display 42 (FIG. 1) to display an image as asub-signal on a sub-screen for PinP (Picture in Picture) or tosplit-display an image.

Furthermore, as the sub-signal, for example, a control signal forcontrolling the display of the main image can be used. In this case, itis possible for the display 42 to control the display of the main imagein response to a control signal as a sub-signal.

When an image (moving image) is used as a sub-signal, in the case thataudio is accompanied with the image, the audio can be assigned totransmission pixel data and transmitted, and can also be transmitted inthe data island period (FIG. 4) similarly to the audio accompanied withthe main image. That is, in HDMI(R), in the data island period (FIG. 4),the audio data of a plurality of audio channels can be transmitted, andthe audio accompanied with the image as the sub-signal can betransmitted using an audio channel that is not used for the transmissionof the audio data accompanied with the main image.

Next, the series of processes of the source signal controller 71 and thesink signal processor 82 can be performed by dedicated hardware and canalso be performed by software. When the series of processes is to beperformed by software, a program forming the software is installed into,for example, a computer, such as a microcomputer, for controlling theHDMI(R) source 53 and the HDMI(R) sink 61.

FIG. 25 shows an example of the configuration of an embodiment of acomputer to which programs for executing the above-described series ofprocesses are installed.

Programs can be recorded in advance in an EEPROM (Electrically ErasableProgrammable Read-only Memory) 205 and a ROM 203 serving as recordingmedia incorporated in the computer.

Alternatively, programs can be temporarily or permanently stored(recorded) on a removable recording medium, such as a flexible disk, aCD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disc, aDVD (Digital Versatile Disc), a magnetic disk, or a semiconductormemory. Such a removable recording medium can be provided in the form ofpackaged software.

In addition to being installed into the computer from the removablerecording medium such as that described above, programs may betransferred in a wireless manner from a download site or may betransferred by wire to the computer via a network, such as a LAN (LocalArea Network) or the Internet. It is possible for the computer toreceive the programs that are transferred in such a manner via aninput/output interface 206 and to install the programs into the EEPROM205 incorporated therein.

The computer has a CPU (Central Processing Unit) 202 incorporatedtherein. The input/output interface 206 is connected to the CPU 202 viaa bus 201, and the CPU 202 loads a program stored in the ROM (Read OnlyMemory) 203 or the EEPROM 205 into a RAM (Random Access Memory) 204 andexecutes the program. As a result, the CPU 202 performs processing inaccordance with the above-described flowcharts or performs processingperformed according to the constructions in the above-described blockdiagrams.

In this specification, the processing steps describing the program bywhich the computer performs various processes do not have to be executedchronologically according to the written orders in flowcharts, and theprocessing steps include processes which are executed in parallel orindividually (for example, parallel processes or object-orientedprocesses).

The programs may be processed by one computer, and may also be processedin a distributed manner by a plurality of computers.

The embodiments of the present invention are not limited to theabove-described embodiments, and various modifications are possiblewithin the spirit and scope of the present invention.

That is, for example, all of the number B1 of bits of the pixel data ofthe main image, the number B2 of bits of the transmission pixel data,and the number B3 of sub-signal assignment bits are not limited to theabove-described values. Furthermore, for example, areas for assigningthe bits Suport_(—)30 bit, Suport_(—)36 bit, and Suport_(—)48 bit, thebits Sub_(—)2 bit, Sub_(—)4 bit, Sub_(—)8 bit, and Sub_Data_Support, thebits SD0, SD1, and SD2 serving as sub-signal information, and the likeare not limited to the above-described areas. In the current HDMI(R),they can be assigned to any areas that are unused (Reserved).

The present invention can be applied to, in addition to HDMI(R), acommunication interface including a transmission apparatus for, aftercapability information indicating the capability of a receivingapparatus is received, transmitting the pixel data of an uncompressedimage for one screen in one direction to the receiving apparatus byusing a differential signal through a plurality of channels fortransmitting data of a fixed number of bits per clock of a pixel clock,in a valid image period that is a period in which a horizontal blankingperiod and a vertical blanking period are excluded from the period fromone vertical synchronization signal to the next vertical synchronizationsignal, and a receiving apparatus for receiving pixel data transmittedusing a differential signal from the transmission apparatus through aplurality of channels after the capability information is transmitted.

Next, a description will be given of a sending method, a transmissionsystem, a transmission method, a transmission apparatus, a receivingmethod, and a receiving apparatus to which the present invention isapplied.

The present invention relates to a sending method and a transmissionsystem that are suitable for use with a digital video/audio input/outputinterface standard called the HDMI (High-Definition MultimediaInterface) standard, and to a transmission method, a transmissionapparatus, a receiving method, and a receiving apparatus for use withthe transmission system.

In recent years, as an interface standard for transmitting uncompresseddigital video data and the like among a plurality of video devices, aninterface standard called the HDMI standard has been developed. The HDMIstandard is a standard for transmitting video data individually in unitsof one pixel as the primary-color data of each color (hereinafterreferred to as a “pixel”). Audio data is also transmitted using atransmission line of the video data in the blanking period of the videodata. For the primary-color data to be transmitted, primary-color data(R data, G data, B data) of 3 channels of additive color mixing of red,green, and blue is transmitted, or luminance and color-differencesignals of Y, Cb, and Cr are transmitted.

Data of one pixel of each color is basically composed of 8 bits. Withregard to synchronization signals such as a horizontal synchronizationsignal and a vertical synchronization signal, they are transmitted at atiming at which each synchronization signal is located. Furthermore, atransmission line for a pixel clock for video data and a transmissionline for control data are also provided.

FIG. 33 shows an overview of an example when primary-color data (R data,G data, B data) is transmitted using the interface of the HDMI standard.Regarding the video data, B data, G data, and R data are individuallytransmitted through three channels, that is, channel 0, channel 1, andchannel 2. In the example of FIG. 33, a period in which data of 4 pixelsof pixels 0, 1, 2, and 3 is transmitted is shown, and the data of 1pixel of each channel is composed of 8 bits.

That is, for the B data (blue data), 8-bit data B0 is transmitted in theperiod of pixel 0 by using channel 0. Hereafter, 8-bit data B1, 8-bitdata B2, and 8-bit data B3 are sequentially transmitted insynchronization with the pixel clock (not shown). For the G data (greendata), 8-bit data G0 is transmitted in the period of pixel 0 by usingchannel 1. Hereafter, 8-bit data G1, 8-bit data G2, and 8-bit data G3are sequentially transmitted in synchronization with the pixel clock.For the R data (red data), 8-bit data R0 is transmitted in the period ofpixel 0 by using channel 2. Hereafter, 8-bit data R1, 8-bit data R2, and8-bit data R3 are sequentially transmitted in synchronization with thepixel clock.

Although not shown in FIG. 33, control data and the pixel clock aretransmitted using another channel. The control data is structured insuch a manner that it can be transmitted from a transmission-sideapparatus (source-side apparatus) for video data to the receiving-sideapparatus (sink-side apparatus) and also can be transmitted from thereceiving-side apparatus (sink-side apparatus) to the transmission-sideapparatus (source-side apparatus). In the source-side apparatus, data isencrypted in units of 8 bits, and in the sink-side apparatus, theencrypted data is decrypted in units of 8 bits.

In the manner described above, the interface of the HDMI standard isstandardized on the assumption that one pixel is sent in units of 8 bitsper color. On the other hand, in recent years, it has been studied toincrease the resolution of color, and it has been proposed that thenumber of bits per color of one pixel be made more than 8. For example,it has been proposed that the number of bits per color of one pixel bemade 10 or 12.

FIG. 34 shows an example of a transmission state in which 10-bit data isassumed to be transmitted for each color of one pixel in the interfaceof the HDMI standard. As has already been described, the HDMI standardis a standard assumed such that data is transmitted using 8 bits as oneunit, 8 bits are transmitted in one pixel clock, and in order totransmit 10-bit data, 2 pixel clocks are necessary. In the example ofFIG. 34, data is arranged in such a manner that data of two pixels istransmitted in three pixel clocks. Phases 0, 1, and 2, shown in FIG. 34,each indicate one cycle of one pixel clock.

The data structure of FIG. 34 will be described. For example, regardingB data, 8 bits among the 10 bits of the pixel 0 are sent in the periodof phase 0 of the channel 0. In the period of phase 1, the remaining 2bits of the pixel 0 are sent, and in the period of the subsequent twobits, dummy data, which is invalid data, is sent. Then, in the period ofthe 4 bits of the second half of the phase 1, 4 bits among the 10 bitsof the next pixel 1 are sent. In the next period of phase 2, theremaining 6 bits of the pixel 1 are sent, and in the period of thesubsequent two bits, dummy data, which is invalid data, is sent.Hereinafter, this arrangement is repeated. For the G data of channel 1and the R data of channel 2, pixel data and dummy data are sent in thesame data arrangement. The period in which the dummy data is arranged isan example, and the dummy data may be arranged in another period. In thecase of the data arrangement shown in FIG. 34, since a period of 1.5pixel clocks is required, the pixel clock needs to have a higherfrequency in a corresponding manner.

As a result of forming the data structure shown in FIG. 34, it ispossible to send pixel data having a large number of bits comparativelyefficiently by using the interface of the HDMI standard that is assumedsuch that data is transmitted using 8 bits as one unit.

In PCT publication WO2002/078336, the details on the HDMI standard havebeen described.

There has been a demand for making more advanced data transmissionpossible by using this type of interface, such as the HDMI standard.That is, the HDMI standard stipulates that video data and audio dataassociated with the video data are transmitted between a video device onthe source side and a video device on the sink side. In addition to atransmission line for transmitting video data and audio data, atransmission line for transmitting control data is also provided, andthe control data can be transmitted using the line for control data.However, there has been a demand for allowing still another piece ofdata to be transmitted simultaneously.

The present invention has been made in view of such points. An object ofthe present invention is to be capable of efficiently performing datatransmission by using a standard in which the number of bits that can betransmitted is determined in a fixed manner, such as the HDMI standard.

The present invention has been configured as described below. When videodata is to be transmitted using a transmission scheme for transmittingvideo data in units of 8-bits in synchronization with a pixel clock froma source-side apparatus to a sink-side apparatus by using individualtransmission lines for each piece of color data or for each of luminanceand color-difference signals, video data of one pixel to be transmittedfrom the source-side apparatus to the sink-side apparatus is made to beof a predetermined number of bits, which is not an integral multiple of8 bits. The video data of a predetermined number of bits, which is notan integral multiple of 8 bits, is transmitted at a timing synchronizedwith a pixel clock. Data differing from the video data of apredetermined number of bits is arranged in a marginal transmissionperiod that occurs in an amount corresponding to the number of bits ofthe difference between the number of bits transmitted in the period ofthe pixel clock assigned to the transmission of one pixel and thepredetermined number of bits, and is transmitted from the source-sideapparatus to the sink-side apparatus.

As a result of the above, it is possible to transmit various kinds ofdata differing from main video data by using a marginal transmissionperiod that occurs in an amount corresponding to the number of bits ofthe difference between the number of bits transmitted in the period ofthe pixel clock assigned to the transmission of one pixel and thepredetermined number of bits.

According to the present invention, it is possible to transmit variouskinds of data differing from main video data by using a marginaltransmission period that occurs in an amount corresponding to the numberof bits of the difference between the number of bits transmitted in theperiod of the pixel clock assigned to the transmission of one pixel andthe predetermined number of bits. Both the transmission of pixel datahaving a number of bits larger than 8 bits and the transmission ofvarious kinds of data differing from main video data can be performed,and thus transmission efficiency is improved. Furthermore, thetransmission mode is a transmission mode in which the transmission unitof 8 bits, which is defined by the transmission standard, is maintained,and encryption and decryption can be performed in a state defined by thestandard.

An embodiment of the present invention will be described below withreference to FIGS. 26 to 32.

In this example, the present invention is applied to a transmissionsystem for transmitting video data and the like from a source-sideapparatus to a sink-side apparatus by using the HDMI standard. FIG. 26shows a system configuration of this example, in which a videorecording/reproduction apparatus 310 that is a source-side apparatus anda television receiver 330 that is a sink-side apparatus are connected toeach other via an HDMI cable 301, so that video data and audio data aretransmitted from the video recording/reproduction apparatus 310 to thetelevision receiver 330. In the following description, regarding theHDMI standard, necessary configuration and the like are described insequence. Basically, the existing HDMI standard is used as it is, andthe configuration of the HDMI cable 301 and the like are the same asthose of the related art.

The video recording/reproduction apparatus 310 will be described first.The video recording/reproduction apparatus 310 includes arecording/reproduction section 311 and can record and reproduce videodata and audio data. As the recording/reproduction section 311, forexample, a hard disk drive (HDD) apparatus can be used. Video dataobtained by reproduction by the recording/reproduction section 311 issupplied to a video processor 312, and audio data obtained byreproduction is supplied to an audio processor 314. The videorecording/reproduction apparatus 310 further includes a tuner 316, andvideo data and audio data obtained by reception by the tuner 316 aresupplied to the video processor 312 and the audio processor 314.

The video processor 312 performs processing for setting video dataobtained by reproduction or reception to be video data for transmission.At this point, the video processor 312 of this example is configured soas to be capable of simultaneously processing video data of two systems,so that video data for a main image and video data for a sub-image canbe generated. Regarding the video data for a main image, for example,one pixel is set as 10-bit data for each color, and regarding the videodata for a sub-image, for example, one pixel is set as 2-bit data foreach color.

The audio processor 314 performs processing for setting audio dataobtained by reproduction or reception to be audio data for transmission.At this point, the audio processor 314 of this example can performprocessing for setting the supplied audio data to be audio data of ageneral data structure for 2-channel audio reproduction and also to beaudio data for which multi-channel reproduction, such as 5.1 channel, isperformed. Audio data for 2-channel reproduction and audio data formulti-channel reproduction can be output simultaneously. The audio datafor multi-channel reproduction may be bit-compressed audio data.

The video data and the audio data output by the video processor 312 andthe audio processor 314 are output to an HDMI transmission processor320. The HDMI transmission processor 320 is a circuit part forperforming transmission processing of the interface of the HDMI standardand is, for example, formed as an integrated circuit. The video data andthe audio data supplied to the HDMI transmission processor 320 aremultiplexed by a multiplexing circuit 321. During the multiplexing, forthe video data of the main image, data of 1 pixel is arranged using theperiod of 1.5 pixel clocks. However, since the transmission of 8 bits ispossible for each channel in the period of 1 pixel clock, thetransmission of 12 bits is possible in the period of 1.5 pixel clocks.In the case of this example, by using a period for 2 bits, which is amarginal period that occurs every 1.5 pixel clocks, other data isarranged by the multiplexing circuit 321.

As the other data, for example, sub-image data generated by the videoprocessor 312 is arranged. As has already been described, the sub-imagedata is uncompressed video data such that 1 pixel is 2 bits per color,and is arranged at the interval of 1 pixel in the marginal period thatoccurs for 2 bits per pixel. However, horizontal synchronization dataand vertical synchronization data in a blanking period are transmittedwith regard to only the main image, and with regard to the sub-imagedata, vertical synchronization data and horizontal synchronization dataexclusively used for the sub-image data are not transmitted. A specificexample of the transmission of data will be described later. As theabove-described other data, audio data for multi-channel reproductionmay be arranged separately every two bits at the intervals of 1.5 pixelclocks. Alternatively, control data having a comparatively large amountof transmission data generated by the controller 315 of the videorecording/reproduction apparatus 310 or auxiliary information may bearranged separately every two bits at the intervals of 1.5 pixel clocks.

2-channel audio data is multiplexed in such a manner as to betransmitted using a blanking period of a channel in which the video datatherefor is transmitted. The processing in which the 2-channel audiodata is arranged in the blanking period and is transmitted is a commontransmission process formulated by the HDMI standard.

Then, the data for transmission, which was multiplexed by themultiplexing circuit 321, is encrypted by an HDCP encryptor 322. TheHDCP encryptor 322 is designed so as to encrypt a channel in which atleast video data is transmitted in accordance with the HDCP(High-bandwidth Digital Content Protection System) standard. Encryptionherein is performed by using 8-bit data of 1 channel as a unit.

The data encrypted by the HDCP encryptor 322 is supplied to thetransmission processor 323. Pixel data of each color is arranged inindividual channels. In a pixel clock channel and a control datachannel, a corresponding clock and corresponding data are arrangedrespectively, and these are sent to the HDMI cable 301 connected to anHDMI terminal 324.

The HDMI cable 301 is connected to an HDMI terminal 341 of a televisionreceiver 330. The data transmitted via the HDMI cable 301 connected tothe HDMI terminal 341 is detected (received) in synchronization with thepixel clock by a transmission processor 342 within the HDMI transmissionprocessor 340. The detected data of each channel is decrypted from theencryption at the time of transmission by an HDCP decryptor 343.Decryption herein is also performed in units of 8 bits per channel.

The decrypted data is supplied to a demultiplexing circuit 344, wherebythe data multiplexed for each channel is separated. For the separationprocess herein, audio data (audio data of 2 channels) arranged in theblanking period of the channel in which a video is transmitted isseparated from the video data (main video data). Furthermore, the dataarranged in the marginal period for two bits, which occurs every 1.5pixel clocks, is also separated from the video data. When the dataarranged in the marginal period is sub-video data, the sub-video data isextracted. When the data arranged in the marginal period ismulti-channel audio data, the multi-channel audio data is extracted.When the data arranged in the marginal period is control data orauxiliary information, the control data or the auxiliary information isextracted.

The main video data and the sub-video data separated by thedemultiplexing circuit 344 is supplied to a video selector/combiner 331.The video selector/combiner 331 selects one of the video images on thebasis of an instruction from a controller 336 of the television receiver330 and supplies the selected video data to the video processor 332. Thevideo processor 332 performs necessary processing on the supplied videodata and supplies it to a display processor 333. The display processor333 performs processing for driving a display panel 360.

The audio data separated by the demultiplexing circuit 344 is suppliedto an audio processor 334, whereby audio processing, such as analogconversion, is performed thereon. The processed output is supplied to anoutput processor 335, whereby processing such as amplification isperformed thereon so as to drive a speaker so that audio is output froma plurality of speakers 351 to 354 connected to an output processor 335.When the audio data supplied to the audio processor 334 is 2-channelaudio data, processing for 2 channels is performed, and when the audiodata is multi-channel audio data, processing for multi-channel audioreproduction is performed.

The control data separated by the demultiplexing circuit 344 is suppliedto the controller 336. The control data can also be sent from thecontroller 336 of the television receiver 330 to the controller 315 onthe video recording/reproduction apparatus 310 side by using a controldata channel.

FIG. 27 shows an example of data structure for each channel throughwhich data is transmitted via the HDMI cable 301 between thetransmission processor 323 of the video recording/reproduction apparatus310 and the transmission processor 342 of the television receiver 330.As shown in FIG. 27, as channels for transmitting video data, threechannels, that is, channel 0, channel 1, and channel 2, are provided andfurthermore, a clock channel for transmitting a pixel clock is provided.Furthermore, a DDC line and a CEC line are provided as control datatransmission channels.

On the transmission side, transmission processors (transmitters) 323 a,323 b, and 323 c are provided within the transmission processor 323 foreach channel for transmitting video data. Also, on the reception side,transmission processors (data receivers) 342 a, 342 b, and 342 c areprovided within the transmission processor 342 for each channel fortransmitting video data.

The configuration of each channel will be described. In the channel 0,pixel data of B data, vertical synchronization data, horizontalsynchronization data, and auxiliary data are transmitted. In the channel1, pixel data of G data, two types of control data (CTL0, CTL1), andauxiliary data are transmitted. In the channel 2, pixel data of R data,two types of control data (CTL2, CTL3), and auxiliary data aretransmitted.

FIG. 28 shows line structure and pixel structure for one frame, which istransmitted in the transmission configuration of this example. Videodata (main video data) transmitted in the case of this example isuncompressed data, and a vertical blanking period and a horizontalblanking period are added thereto. More specifically, in the example ofFIG. 28, a video area (area shown as an active video area) to bedisplayed is set to be pixel data of 480 lines× . . . 720 pixels, andthe number of lines and the number of pixels containing up to theblanking period are set to be 525 lines and 858 pixels, respectively.The area indicated by double hatching (the oblique lines in the rightdirection and in the left direction) in the blanking period is a period,in which auxiliary data can be added, which is called a data island.

Next, a description will be given, with reference to FIG. 29, of a statein which data is transmitted using channel 0, channel 1, and channel 2through which pixel data is transmitted in the transmissionconfiguration of this example. In the example of FIG. 29, data isarranged such that data of two pixels is transmitted in 3 pixel clocks.As data arranged in a marginal period for 2 bits, which occurs every 1.5pixel clocks, sub-image data is used as an example. Phases 0, 1, and 2shown in FIG. 29 each indicate one cycle of one pixel clock.

The data structure of FIG. 29 will be described. For example, regardingthe B data, 8 bits among the 10 bits of the pixel 0 of the main imagedata in the period of phase 0 of the channel 0 are sent, the remaining 2bits of the pixel 0 of the main image data are sent in the period atphase 1, and one pixel of B data of the sub-image data is sent in theperiod of the subsequent 2 bits.

Then, in the 4-bit period of the second half period of phase 1, 4 bitsamong the 10 bits of the next pixel 1 of the main image data are sent.In the period of the next phase 2, the remaining 6 bits of the pixel 1of the main image data are sent, and in the period of the subsequent 2bits, one pixel of the B data of the sub-image data is sent.Hereinafter, this arrangement is repeated. Regarding the G data of thechannel 1 and the R data of the channel 2, the pixel data of the mainimage data and the pixel data of the sub-image are sent in the same dataarrangement. In FIG. 29, data B0, G0, R0, B1, G1, and R1 each indicatepixel data of the three primary colors of the main image. Data BS0, GS0,RS0, BS1, GS1, and RS1 each indicate pixel data of the three primarycolors of the sub-image.

FIG. 30 shows another example of data structure. In this example, whencompared with the example of FIG. 29, as the period of phase 1, in theperiod of the first 2 bits, the remaining 2 bits of the pixel data ofthe main image of the pixel 0, which continues from the period of theprevious phase 0, are sent. Next, 2-bit pixel data of the pixel 0 of thesub-image is sent, and furthermore, 2-bit pixel data of the pixel 1 ofthe sub-image is sent. Then, in the period of the final two bits atphase 1, the first 2 bits of the pixel data of the main image of thepixel 1 are sent, and at phase 2, the pixel data of the main image ofthe remaining 8 bits of pixel 1 is sent. In the manner described above,in the example of FIG. 30, the position at which the pixel data of thesub-image is arranged differs from that in the example of FIG. 29.

In the examples of FIGS. 29 and 30, as data other than the pixel data ofthe main image, the pixel data of the sub-image is used. Also, whenother data, such as multi-channel audio data and control data, is to bearranged, it may be arranged at identical positions. As control data,for example, luminance control data for backlight required by a displaypanel may be sent.

FIG. 31 shows an example in which multiplexing data example isinstructed from the source side to the sink side by using data calledVSDB, which is data that specifies the structure of transmission data,when multiplexing for arranging data other than pixel data of a mainimage is performed in the manner described above. Data of VSDB is datatransmitted using a DDC line (FIG. 27). In the case of the VSDB of thisexample, the data of the 6th byte indicates how many bits one pixel iscomposed of. In the case of this example, data of a total of 30 bitssuch that one pixel is 10 bits per color is shown. Then, thepresence/absence of the sub-image is shown. In place of the data for thepresence/absence of the sub-image, the presence/absence of addition ofmulti-channel audio or the presence/absence of addition of control datamay be shown.

The controller 336 (FIG. 26) of the sink-side apparatus (televisionreceiver 330) determines what kind of format the sub-image istransmitted by making a determination as to the data of the VSDB, andallows the demultiplexing circuit 344 or the like to perform processing,such as separation and decoding of the data of the received sub-image,so that display using the sub-image is correctly performed.

As data regarding the sub-image sent using VSDB, more detailed data,such as the number of pixels of the sub-image, may be sent. For example,when there are 4 types of formats A, B, C, and D as the format of thesub-image, the transmission may be configured such that which one of the4 types the format is by using the low-order 4 bits of the 6th bytedata. The details of the formats A, B, C, and D may be transmittedseparately and may be reported to the sink-side apparatus. For example,the format A may be such that the sub-image has the same number ofpixels, one pixel of each color is 2 bits, and the main image and thesub-image have the same frame rate. The format B may be such that thesub-image has a number of pixels of ¼ that of the main image, one pixelof each color is 8 bits, and the main image and the sub-image have thesame frame rate. The details of such data structure of the sub-image maybe transmitted and reported.

At this point, the data of VSDB is used to instruct a multiplexing dataexample from the source side. Alternatively, a device on the sink sidemay send identical data in order to show the capability of data that canbe received (a display process capability). That is, when connecteddevices mutually authenticate, the controller of the sink-side apparatusshows its own display process capability to the source side by using thedata of the VSDB (or other data). The controller on the source sideconstructs data so as to transmit the data of the sub-image at a formatmatching the capability. As a result of the above, a state ofappropriate sub-image data transmission is reached.

The configuration in which data regarding a sub-image is transmittedusing the data of the VSDB transmitted through a DDC line is an example.In addition, similar data, which is transmitted between the source-sideapparatus and the sink-side apparatus, may be transmitted using anotherdata period. For example, in a portion of a period of a data island inthe blanking period shown in FIG. 28, additional information indicatingthat data containing a sub-image has been transmitted may be arranged.

At this point, the amount of data, such as a sub-image, that can betransmitted, will be studied. For example, when the frequency of theclock channel is approximately 225 MHz, data up to approximately 900Mbps can be transferred on the basis of 225 MHz*(3 ch*8 bits)*((12bits−10 bits)/12 bits)=900 Mbps. Furthermore, when a study is made onthe basis of the number of pixels of a main image, in the case that themain image is 1920 pixels× . . . 1080 p at 60 Hz, the sub-video imagebecomes, for example, 1920 pixels× . . . 1080 p at 60 Hz at 2̂6=64colors, and the sub-image is sent in synchronization with the main videoimage. In another example, when the main video image is 1920 pixels× . .. 1080 p at 60 Hz, if the resolution of the sub-image is made to be 960pixels× . . . 540 p half that the length and width of the main image, asub-video image such that one pixel is 8 bits is obtained for eachcolor. Furthermore, a sub-video image of an SD resolution of 720 pixels×. . . 480 p at 60 Hz such that one pixel is 12 bits for each color canbe sent in synchronization with the main video image. In the mannerdescribed above, for the sub-image, the color can be made to berepresented by a larger number of bits, and the combination of thenumbers of pixels can be changed as desired in a range not exceeding thetransmission speed of data (dummy data in FIG. 34) that is left over inthe transmission of the main image.

FIG. 32 shows an example of a main image and a sub-image. In thisexample, as a main image displayed in the television receiver 330, anaerial photograph (satellite photograph) image 361 of a specific placeis shown, and a map image of that place is shown as a sub-image 362. Bysending and displaying the main image and the sub-image that are relatedto each other in the manner described above, images can be useddifferently. In the case of this example, since the main image and thesub-image can be transmitted in such a manner that the bit positions arecompletely synchronized, synchronization data of the main image can becommonly used, and thus efficient transmission is possible.

By applying the transmission process of this example in the mannerdescribed above, the number of bits of the main image data can beincreased to the number of bits in one transmission unit, and also,various kinds of data, such as a sub-image, can be transmitted using bitpositions that are left over at that time. Thus, it is possible toachieve both a larger number of bits and improved transmissionefficiency.

In the embodiment described thus far, an example has been described inwhich data of 10 bits per pixel is transmitted. When data of 12 bits, 14bits, or the like differing from the number of bits (here, 8 bits) inbasic transmission units is to be transmitted, other data such as asub-image may be transmitted in the period of the bits that are leftover in that case.

Furthermore, the present invention may be applied to a format at whichtransmission is possible at another number of bits, such as a format atwhich one pixel can be transmitted in units of 16 bits. Also, withregard to a unit for encryption and decryption, the present inventionmay be applied to processing performed by using another number of bits,such as 16 bits, as a unit.

The embodiment has been described on the assumption of the interface ofthe HDMI standard. The present invention can be applied to other similartransmission standards.

HDMI allows any video format timing to be transmitted and displayed. Tomaximize interoperability between products, common DTV formats have beendefined. These video format timings define the pixel and line counts andtiming, synchronization pulse position and duration, and whether theformat is interlaced or progressive. HDMI also allows vendor-specificformats to be used.

In HDMI(R), the video pixels carried across the link are in one of threedifferent pixel encodings: RGB 4:4:4, YC_(B)C_(R) 4:4:4 or YC_(B)C_(R)4:2:2.

The HDMI “source” determines the pixel encoding and video format of thetransmitted signal based on the characteristics of the source video, theformat and pixel encoding conversions possible at the “source”, and theformat and pixel encoding capabilities and preferences of the “sink”.

In HDMI(R), in order to provide maximum compatibility between video“sources” and “sinks”, specific minimum requirements have been specifiedfor “sources” and “sinks”.

In HDMI(R), some of support necessary conditions (i) to (vii) shownbelow are added to those specified in CEA-861-D.

(i) An HDMI “source” supports at least one of the following video formattimings.

640×480p @ 59.94/60 Hz

720×480p @ 59.94/60 Hz

720×576p @ 50 Hz

(ii) An HDMI “source” that is capable of transmitting any of thefollowing video format timings using any other component analog oruncompressed digital video output, is capable of transmitting that videoformat timing across the HDMI interface.

1280×720p @ 59.94/60 Hz

1920×1080i @ 59.94/60 Hz

720×480p @ 59.94/60 Hz

1280×720p @ 50 Hz

1920×1080i @ 50 Hz

720×576p @ 50 Hz

(iii) An HDMI “sink” that accepts 60 Hz video formats supports the640×480p @ 59.94/60 Hz and 720×480p @ 59.94/60 Hz video format timings.

(iv) An HDMI “sink” that accepts 50 Hz video formats supports the640×480p @ 59.94/60 Hz and 720×576p @ 50 Hz video format timings.

(v) An HDMI “sink” that accepts 60 Hz video formats, and that supportsHDTV capability, supports 1280×720p @ 59.94/60 Hz or 1920×1080i @59.94/60 Hz video format timings.

(vi) An HDMI “sink” that accepts 50 Hz video formats, and that supportsHDTV capability, supports 1280×720p @ 50 Hz or 1920×1080i @ 50 Hz videoformat timings.

(vii) An HDMI “sink” that is capable of receiving any of the followingvideo format timings using any other component analog or uncompresseddigital video input, is capable of receiving that format across the HDMIinterface.

1280×720p @ 59.94/60 Hz

1920×1080i @ 59.94/60 Hz

1280×720p @ 50 Hz

1920×1080i @ 50 Hz

During the “data island” period, HDMI carries HSYNC and VSYNC signalsusing encoded bits on “channel” 0. During “video data” periods, HDMIdoes not carry HSYNC and VSYNC and the “sink” should assume that thesesignals remain constant. During “control” periods, HDMI carries HSYNCand VSYNC signals through the use of four different control characterson TMDS “channel” 0.

Only pixel encodings of RGB 4:4:4, YC_(B)C_(R) 4:2:2, and YC_(B)C_(R)4:4:4 (as specified in Section 6.5) may be used on HDMI.

All HDMI “sources” supports either YC_(B)C_(R) 4:2:2 or YC_(B)C_(R)4:4:4 pixel encoding whenever that device is capable of transmitting acolor-difference color space across any other component analog ordigital video interface except where that device would be required toconvert RGB video to YC_(B)C_(R) in order to meet this requirement.

All HDMI “sinks” is capable of supporting both YC_(B)C_(R) 4:4:4 andYC_(B)C_(R) 4:2:2 pixel encoding when that device is capable ofsupporting a color-difference color space from any other componentanalog or digital video input.

If an HDMI “sink” supports either YC_(B)C_(R) 4:2:2 or YC_(B)C_(R)4:4:4, then both are supported.

An HDMI “source” may determine the pixel-encodings that are supported bythe “sink” through the use of the E-EDID. If the “sink” indicates thatit supports YC_(B)C_(R)-formatted video data and if the “source” candeliver YC_(B)C_(R) data, then it can enable the transfer of this dataacross the link.

HDMI “sources” and “sinks” may support color depths of 24, 30, 36 and/or48 bits per pixel. All HDMI “sources” and “sinks” supports 24 bits perpixel.

Color depths of 30, 36, and 48 bits greater than 24 bits are defined as“deep color” modes. If an HDMI “source” or “sink” supports any “deepcolor” mode, it supports 36-bit mode though all “deep color” modes areoptional.

For each supported “deep color” mode, RGB 4:4:4 is supported andoptionally YC_(B)C_(R) 4:4:4 may be supported. YC_(B)C_(R) 4:2:2 is notpermitted for any “deep color” mode.

An HDMI “sink” supports all EDID-indicated “deep color” modes on allEDID-indicated video formats except if that combination exceeds theMax_TMDS_Clock indication.

An HDMI “source” does not send any “deep color” mode to a “sink” thatdoes not indicate support for that mode.

All specified video line pixel counts and video field line counts (bothactive and total) and HSYNC and VSYNC positions, polarities, anddurations is adhered to when transmitting a specified video formattiming.

For example, if a “source” is processing material with fewer activepixels per line than required (i.e. 704 pixels vs. 720 pixels forstandard definition MPEG2 material), it may add pixels to the left andright of the supplied material before transmitting across HDMI. AVI barinfo may need to be adjusted to account for these added pixels.

Detailed timing is found in CEA-861-D or a later version of CEA-861 forthe following video format timings.

The primary video format timings are as follows.

640×480p @ 59.94/60 Hz

1280×720p @ 59.94/60 Hz

1920×1080i @ 59.94/60 Hz

720×480p @ 59.94/60 Hz

720(1440)×480i @ 59.94/60 Hz

1280×720p @ 50 Hz

1920×1080i @ 50 Hz

720×576p @ 50 Hz

720(1440)×576i @ 50 Hz

The secondary video format timings are as follows.

720(1440)×240p @ 59.94/60 Hz

2880×480i @ 59.94/60 Hz

2880×240p @ 59.94/60 Hz

1440×480p @ 59.94/60 Hz

1920×1080p @ 59.94/60 Hz

720(1440)×288p @ 50 Hz

2880×576i @ 50 Hz

2880×288p @ 50 Hz

1440×576p @ 50 Hz

1920×1080p @ 50 Hz

1920×1080p @ 23.98/24 Hz

1920×1080p @ 25 Hz

1920×1080p @ 29.97/30 Hz

2880×480p @ 59.94/60 Hz

2880×576p @ 50 Hz

1920×1080i (1250 total) @ 50 Hz

720(1440)×480i @ 119.88/120 Hz

720×480p @ 119.88/120 Hz

1920×1080i @ 119.88/120 Hz

1280×720p @ 119.88/120 Hz

720(1440)×480i @ 239.76/240 Hz

720×480p @ 239.76/240 Hz

720(1440)×576i @ 100 Hz

720×576p @ 100 Hz

1920×1080i @ 100 Hz

1280×720p @ 100 Hz

720(1440)×576i @ 200 Hz

720×576p @ 200 Hz

Next, a description will be given of pixel repetition.

Video formats whose unique pixel rate is 25M pixels/sec or less needspixel repetition because they are sent via the TMDS link. At the videoformat timings of 720×480i and 720×576i, pixels are always repeated.

Furthermore, the HDMI(R) “source” indicates use of pixel repetitionusing a “pixel-repetition field in AVI InfoFrame. This field indicatesthe number of pixel repetitions, each of which is unique, which istransmitted to the HDMI(R) “sink”. In non-repeated formats, this valueis set to 0.

Regarding pixel-repeated formats, this value indicates the number ofpixels that can be discarded by the “sink” without losing real imagecontent.

The “source” always indicates a pixel repetition count that is usedcorrectly. The use of the “pixel-repetition field is optional withregard to the HDMI(R) “sink”. The use of the pixel repetition isdescribed in detail in CEA-861-D.

There are three types of pixel coding that can be sent via an HDMI(R)cable: YC_(B)C_(R) 4:4:4, YC_(B)C_(R) 4:2:2, and RGB 4:4:4. Whichevercoding is used, this conforms to one of the methods described below.

Four color depths, that is, 24, 30, 36, and 48 bits, are supported perpixel. In the depths (the “deep color” mode) of 30, 36, and 48 bits,which are greater than 24 bits, only RGB 4:4:4 and YC_(B)C_(R) 4:4:4 arepermitted.

FIG. 35 shows default coding, that is, RGB 4:4:4 for a 24-bit colordepth.

The R, G, and B components of the first pixel for a specific line ofvideo are transferred in the first pixel in the video data periodfollowing a guard band character.

FIG. 36 shows a signal mapping and a timing for transferring 24-bitYC_(B)C_(R) 4:2:2 data in HDMI(R).

Since 4:2:2 data needs only two components per pixel, it is possible toassign more bits to each component. In FIG. 36, 24 bits that can be usedare divided into 12 bits for the Y component and are divided into 12bits for the C component.

YC_(B)C_(R) 4:2:2 pixel coding in HDMI(R) resembles much the standardITU-R BT.601. The high-order 8 bits of a Y sample are mapped to 8 bitsof “channel” 1, and the low-order 4 bits are mapped to the low-order 4bits of “channel” 0. When 12 bits or less are used, effective bits arejustified to the left (that is, MSb=MSb) by embedding 0 to the LSb orlower bits.

The first pixel transmitted within the “video data period” containsthree components Y0, Cb0, and Cr0. Y0 and Cb0 components are transmittedin the first pixel period, and Cr0 is transmitted in the second pixelperiod. The second pixel period also contains only the component Y1 ofthe second pixel. In the manner described above, through the link, oneC_(B) sample is sent at intervals of two pixels, and one Cr sample issent at intervals of two pixels.

The two components (C_(B) and C_(R)) are multiplexed and transmitted inthe same signal path of the link.

Furthermore, in the third pixel, this process is repeated on the thirdpixel to be transmitted using the Y and C_(B) components. In the nextpixel period, the C_(R) component of the third pixel and the Y componentof the fourth pixel follow this.

That is, with regard to the image of YC_(B)C_(R) 4:2:2, each pixel has Ycomponent, and one pixel within the two pixels has C_(B) component andC_(R) component at intervals of two pixels. In HDMI(R), 12 bits areassigned to each of the Y component, the C_(B) component, and the C_(R)component of the pixel data of an image of YC_(B)C_(R) 4:2:2.

Then, in HDMI(R), as shown in FIG. 36, with regard to the pixel data ofYC_(B)C_(R) 4:2:2, the low-order 4 bits (bits 3−0) within the Ycomponent of 12 bits of one pixel and the low-order 4 bits (bits 3−0) ofone of the C_(B) components and the C_(R) components of 12 bits of onepixel can be transmitted through the TMDS channel #0 per clock of thepixel clock. Furthermore, through the TMDS channel #1, the high-order 8bits (bits 11−4) within the Y component of 12 bits of one pixel istransmitted. Also, through the TMDS channel #2, the high-order 8 bits(bits 11−4) of one of the C_(B) component and the C_(R) component of 12bits of one pixel can be transmitted.

That is, with respect to the pixel data of YC_(B)C_(R) 4:2:2, the Ycomponent of 12 bits of one pixel is transmitted per clock of the pixelclock. Furthermore, one of the C_(B) component and the C_(R) componentof 12 bits of one pixel is transmitted per clock of one of two pixelclocks, and the other component is transmitted per remaining clock.

FIG. 37 shows a signal mapping and a timing for transferring 24-bitYC_(B)C_(R) 4:4:4 data in HDMI(R).

Next, a description will be given of packing of deep color pixels.

In the color depth of 24 bits/pixel, pixels are sent at the rate of onepixel per TMDS clock. In a color depth deeper than this, the TMDS clockis executed earlier than the source pixel clock and provides an extrabandwidth to additional pixels. Furthermore, the TMDS clock rateincreases at the ratio of the pixel size to 24 bits.

The TMDS clock in each bit mode is shown in the following.

(i) 24-bit mode: TMDS clock=1.0× pixel clock (1:1)

(ii) 30-bit mode: TMDS clock=1.25× pixel clock (5:4)

(iii) 36-bit mode: TMDS clock=1.5× pixel clock (3:2)

(iv) 48-bit mode: TMDS clock=2.0× pixel clock (2:1)

When operating in the “deep color” mode, all the video data (pixels) andsignals (HSYNC, VSYNC, DE transition) are categorized as a series ofpack formats “pixel groups”. Each of them has the same number of pixels,and needs the same number of TMDS clocks for the purpose oftransmission. In each TMDS clock, the “fragment of one pixel group” istransmitted. The number of pixels per group and the number of fragmentsper group are determined by the pixel size.

In the following, the number of pixels per group and the number offragments per group in each bit mode are shown.

(i) 24-bit mode: 1 pixel/group, 1 fragment/group

(ii) 30-bit mode: 4 pixels/group, 5 fragments/group

(iii) 36-bit mode: 2 pixels/group, 3 fragments/group

(iv) 48-bit mode: 1 pixel/group, 2 fragments/group

During the active video period, the input pixel data is packed to thesegroups. During the blanking period, HSYNC and VSYNC are packed to thesesame groups. In the manner described above, all the video-relatedprotocol elements are sent in direct proportion to the pixel clock. Inthe manner described above, it is ensured that there is no change in therelation between the pixel clock and the pixel data and between DEtransition and HSYNC or VSYNC transition. This makes it possible to besupported in 24 bits/pixel, which is supported equally at another pixelsize.

All other HDMI(R) protocol elements are not affected by “deep color”pixel packing. “Data islands”, “video guard bands”, and “preamble” occurin the same manner as that in a normal (24-bit) mode. That is, each“preamble” is 8 TMDS clocks, each “data island” is 32 TMDS clocks, andeach “guard bandwidth” is 2 TMDS clocks.

In the manner described above, the pixel group is composed of 1, 2, or 4pixels. Each pixel group is divided into 1, 2, 3, or 5 pixel fragments,in which one fragment is transmitted per TMDS clock.

In each TMDS character period (1 TMDS clock) within a stream to betransmitted, the “fragment of a single pixel group” is sent, therebyshowing a specific “packing phase” of the group.

In order to synchronize the unpacking state of the pixel with the pixelpacked state of the source, the “sink” needs to make a determination asto which character within a character stream indicates the start orphase 0 of a new group. In order to achieve this, the source sends apacket indicating the packing phase of a specific pixel. This packet issent at least once per video field and indicates the then-currentpacking state. By using the data, the sink determines the first startpoint of each new group, confirms that synchronization continues byusing periodic updating, or recovers from gross errors in the link.

FIG. 38 shows all “pixel codings” for all color depths.

In FIG. 38, packing of each phase is shown with regard to each mode. Thepacking phase for active video is identified as “mPn” (10P0, 10P1,etc.), and the packing phase of blanking is identified as “mCn” (10C0,10C1, etc.).

During the blanking, one HSYNC value and one VSYNC value are sent perpixel clock in each “pixel group”. This is for providing HSYNC and VSYNCslots, the number of which is greater by one than the necessary numberper group (for example, 5 TMDS clocks with respect to 4 pixels). Forthis reason, the HSYNC and the VSYNC values are simply repeated in thefinal TMDS clock of the group.

FIGS. 39 to 43 show a group size and the sequence of HSYNC and VSYNCtransmission within the group with regard to a 24-bit mode, a 30-bitmode, a 30-bit mode, the remainder of a 36-bit mode, and a 48-bit mode.

In FIGS. 39 to 43, source HSYNC/VSYNC values of each pixel are labeledas S, T, U, and V (as necessary). The source HSYNC/VSYNC value S is theleftmost (earliest) code within the group.

In a 30-bit mode, if the “video data period” ends before the pixel groupboundary, the remaining fragments are filled using one or more stepsfrom the 10PCn sequence shown in FIG. 41 until the group boundary isreached (step 10PC4). After that, the normal sequence is used (steps10Cn).

FIG. 41 shows the remainder of a 30-bit mode (the falling edge of DEoccurs in an intermediate group). “10PCn” shown in FIG. 41 refers to abridge state regarding the transition from 10Pn to 10C0.

When in a “deep color” mode, the “source” and “sink” each record thepacking phase of the last pixel character of a “video data” period.

The “source” occasionally sends a “general control packet (GCP)”communicating the current color depth and the packing phase of the lastpixel character sent prior to the GCP. This data is valid in the GCPwhenever CD (CD0, CD1, CD2, CD3) is nonzero.

Whenever the “sink” receives a GCP with non-zero CD data, it shouldcompare the receiver's own color depth and phase with the CD data. Ifthey do not match, the “sink” should adjust its color depth and/or phaseto match the CD data.

When transmitting “deep color”, the “source” sends a “general controlpacket (GCP)” with an accurate CD field indicating the current colordepth and with the PP field (PP0, PP1, PP2, PP3) indicating the packingphase of the last pixel character (within the last “video data period”)sent prior to the GCP. “Sources” only send GCPs with non-zero CD to“sinks” that indicate support for “deep color”, and only select colordepths supported by the “sink”.

Once a “source” sends a GCP with non-zero CD to a sink, it shouldcontinue sending GCPs with non-zero CD at least once per video fieldeven if reverting to 24-bit color, as long as the “sink” continues tosupport “deep color”.

When the “sink” does not receive a GCP together with a non-zero CD for 4or more consecutive video fields, the “sink” should need to exit thedeep color mode (revert to 24-bit color).

FIG. 44 shows color depth values (CD fields) of SB1.

As shown in FIG. 44, when the CD is 0, information on the color depth isnot indicated. PP is set to 0.

When the CD is other than 0, the color depth is displayed, and thepacking phase bit (PP) is valid.

When the “sink” does not indicate support for the “deep color”, a CDfield of 0 (the “color depth” is not indicated) is used. This value mayalso be used in the “deep color” mode to transmit a GCP indicating only“non-deep color” information (e.g., AVMUTE).

When the CD field indicates 24 bits per pixel, the PP field is invalidand should be ignored by the “sink”.

In the “pixel packing phase” field (PP) of SB1, when the CD field is 0,the PP field is also set to 0. When the CD field is not 0, the PP fieldindicates the packing phase of the final fragment of the most recent“video data period” (prior to the GCP message).

FIG. 45 shows a specific PP value regarding each packing phase shown ina packing phase table in an early period.

Since phase 0 always represents only part of the first pixel of thegroup, no “video data period” will end at phase 0. Therefore, Packingphase 0 does not need to be indicated using the PP bits. If the activevideo ends after the first pixel, then the final phase will be phase 1,containing the last bits of the first pixel.

If the transmitted video format has timing such that the phase of thefirst pixel of every “video data period” corresponds to pixel packingphase 0 (e.g., 10P0, 12P0, 16P0), the “source” may set the Default_Phasebit in the GCP. The “sink” may use this bit to optimize its filtering orhandling of the PP field.

Next, a description will be given of a Default_Phasefield of GCP SB2.

When Default_Phase is 0, the PP bits may vary and the first pixel ofeach “video data period” may vary.

When Default_Phase is 1, (i) to (iv) described below are true.

(i) The first pixel of each “video data period” always has a pixelpacking phase of 0 (10P0, 12P0, 16P0).

(ii) The first pixel following each “video data period” has a pixelpacking phase of 0 (10C0, 12C0, 16C0).

(iii) The PP bits are constant for all GCPs and will be equal to thelast packing phase (10P4, 12P2, 16P1).

(iv) The first pixel following every transition of HSYNC or VSYNC has apixel packing phase of 0 (10C0, 12C0, 16C0).

Next, the above-described pixel repetition will be described further.

During pixel-doubling (Pixel_Repetition_Count=1), all of the data sentacross during the first pixel period will be repeated during the secondpixel period. The third pixel period will then represent the secondactual pixel and so on.

FIG. 46 shows YC_(B)C_(R) 4:2:2 of pixel-doubling.

Pixel repetition is permitted in conjunction with “deep color modes”.The source replicates the pixels as described above prior to deep colorpacking into multiple fragments.

Next, a description will be given of video quantization ranges.

Black and white levels for video components are either a “full Range” ora “limited range”. YC_(B)C_(R) components are always a limited rangewhile RGB components may be either a full range or a limited range.While using RGB, the limited range is used for all video formats definedin CEA-861-D, with the exception of VGA (640×480) format, which requiresthe full range.

FIG. 47 shows a video color component gamut.

The component gamut regarding xvYCC has been described in IEC 61966-2-4.

Next, a description will be given of colorimetry. “Sources” willtypically use the specific default colorimetry for the video formatbeing transmitted. If no colorimetry is indicated in the AVI InfoFrame'sC field (C1, C0) then the colorimetry of the transmitted signal matchesthe default colorimetry for the transmitted video format.

The default colorimetry for all 480-line, 576-line, 240-line, and288-line video formats described in CEA-861-D is based on SMPTE 170M.

The default colorimetry of the high-definition video formats (1080i,1080p, and 720p) described in CEA-861-D is based on ITU-R BT.709-5.

The default colorimetry of the other video formats is sRGB.

Next, a description will be given of applicable colorimetry standards.

For any video categorized as SMPTE 170M, ITU-R BT.601-5 Section 3.5 isused for any color space conversion needed in the course of processing.

The encoding parameter values are as defined in Table 3 of ITU-RBT.601-5.

For any video categorized as ITU-R BT.709, Part 1, Section 4 of thatdocument is used for any color space conversion needed in the course ofprocessing.

The digital representation is as defined in Part 1, Section 6.10 ofITU-R BT.709-5.

IEC 61966-2-4 (xvYCC) defines the “Extended-gamut YCC color space forvideo applications”. It is based on the YCC color encoding described inITU-R BT.709-5, but extends its definition to a much wider gamut.

xvCC601 is based on colorimetry defined in ITU-R BT.₆₀₁, and xvYCC₇₀₉ isbased on colorimetry defined in ITU-R BT.709. The details have beendescribed in Chapter 4.3 of IEC 61966-2-4.

Any “source” transmission of xvYCC video (either xvYCC₆₀₁ or xvYCC₇₀₉)is accompanied by the transmission of valid gamut boundary metadata.

If the attached “sink” does not support xvYCC or “gamut metadatapackets”, then the source should not transmit xvYCC-encoded video anddoes not indicate xvYCC₆₀₁ or xvYCC₇₀₉ in the AVI InfoFrame.

Next, a description will be given of gamut-related metadata.

HDMI(R) has capability of sending description of the video gamutboundary by using a “gamut metadata packet”.

Furthermore, the “sink” sets one or more bits of MD0, MD1, and the likein a “colorimetry data block”, thereby indicating support for a specifictransmission profile.

When the EDID of the attached “sink” does not contain a “colorimetrydata block”, the “source” does not transmit a “gamut metadata packet”.Note should be taken as to the fact that the xvYCC colorimetry requiresthat the gamut metadata be transmitted.

FIGS. 48 to 51 are state machine diagrams for each mode of a 24-bitmode, a 30-bit-mode, a 36-bit mode, and a 48-bit mode.

For each mode, the source sequence starts at phase 0 and is incrementedthrough each phase. When DE=1 (pixel data can be used), a pixel datafragment mCn is transmitted. When DE=0, a blanking fragment mCn istransmitted.

Next, a description will be given of recommended N and expected CTSvalues.

FIGS. 52 to 57 show recommended values of N of the standard pixel clockrate in a “deep color” mode.

FIG. 52 shows recommended N and expected CTS values of bits/pixel for 32kHz. FIG. 53 shows recommended N and expected CTS values of 36bits/pixel for 44.1 kHz and multiples. FIG. 54 shows recommended N andexpected CTS values of 36 bits/pixel for 48 kHz and multiples.

FIG. 55 shows recommended N and expected CTS values of bits/pixel for 32kHz. FIG. 56 shows recommended N and expected CTS values of 48bits/pixel for 44.1 kHz and multiples. FIG. 57 shows recommended N andexpected CTS values of 48 bits/pixel for 48 kHz and multiples.

The “source” having a non-interferential clock is recommended to usevalues shown for TMDS clocks of the “others”

1. A communication system comprising: a transmission apparatus for,after capability information indicating the capability of a receivingapparatus is received, transmitting pixel data of an uncompressed imagefor one screen in one direction to the receiving apparatus by using adifferential signal through a plurality of channels for transmittingdata of a fixed number of bits per clock of a pixel clock in a validimage period that is a period in which a horizontal blanking period anda vertical blanking period are excluded from a period from one verticalsynchronization signal to the next vertical synchronization signal; andthe receiving apparatus for receiving the pixel data transmitted using adifferential signal through the plurality of channels from thetransmission apparatus after the capability information is transmitted,wherein the transmission apparatus includes transmission means fortransmitting pixel data to which a number of bits, which is greater thanthe fixed number of bits, is assigned, in one direction to the receivingapparatus by using a differential signal through the plurality ofchannels by adjusting the frequency of the pixel clock; sub-signalreception capability/incapability determination means for determiningwhether or not the receiving apparatus can receive a sub-signal on thebasis of the capability information; sub-signal addition means foradding the sub-signal to pixel data of a main image composed of pixeldata whose number of bits is smaller than that of transmission pixeldata that is pixel data to be transmitted by the transmission means whenthe receiving apparatus can receive the sub-signal, thereby constructingthe transmission pixel data; and information transmission control meansfor allowing sub-signal information indicating whether or not thesub-signal is contained in the transmission pixel data transmitted inthe valid image period immediately after the vertical blanking period,to be transmitted in the vertical blanking period, and wherein thereceiving apparatus includes receiving means for receiving thetransmission pixel data transmitted using a differential signal throughthe plurality of channels; sub-signal presence/absence determinationmeans for determining whether or not the sub-signal is contained in thetransmission pixel data transmitted in a valid image period immediatelyafter the vertical blanking period on the basis of the sub-signalinformation transmitted in the vertical blanking period; and separationmeans for separating the sub-signal from the transmission pixel datawhen the sub-signal is contained in the transmission pixel data.
 2. Atransmission apparatus for, after capability information indicating thecapability of a receiving apparatus is received, transmitting pixel dataof an uncompressed image for one screen in one direction to thereceiving apparatus by using a differential signal through a pluralityof channels for transmitting data of a fixed number of bits per clock ofa pixel clock in a valid image period that is a period in which ahorizontal blanking period and a vertical blanking period are excludedfrom a period from one vertical synchronization signal to the nextvertical synchronization signal, the transmission apparatus comprising:transmission means for transmitting pixel data to which a number ofbits, which is greater than the fixed number of bits, is assigned, inone direction to the receiving apparatus by using a differential signalthrough the plurality of channels by adjusting the frequency of thepixel clock; sub-signal reception capability/incapability determinationmeans for determining whether or not the receiving apparatus can receivea sub-signal on the basis of the capability information; sub-signaladdition means for adding the sub-signal to pixel data of a main imagecomposed of pixel data whose number of bits is smaller than that oftransmission pixel data that is pixel data to be transmitted by thetransmission means when the receiving apparatus can receive thesub-signal, thereby constructing the transmission pixel data; andinformation transmission control means for allowing sub-signalinformation indicating whether or not the sub-signal is contained in thetransmission pixel data transmitted in the valid image periodimmediately after the vertical blanking period, to be transmitted in thevertical blanking period.
 3. The transmission apparatus according toclaim 2, wherein the transmission means determines the number of bits ofthe transmission pixel data on the basis of the capability information,and adjusts the frequency of the pixel clock on the basis of the numberof bits of the transmission pixel data.
 4. A communication method foruse with a transmission apparatus for transmitting pixel data of anuncompressed image for one screen in one direction to a receivingapparatus by using a differential signal through a plurality of channelsfor, after capability information indicating the capability of thereceiving apparatus is received, transmitting data of a fixed number ofbits per clock of a pixel clock in a valid image period that is a periodin which a horizontal blanking period and a vertical blanking period areexcluded from a period from one vertical synchronization signal to thenext vertical synchronization signal, the transmission apparatusincluding transmission means for transmitting pixel data to which anumber of bits, which is greater than the fixed number of bits, isassigned, in one direction to the receiving apparatus by using adifferential signal through the plurality of channels by adjusting thefrequency of the pixel clock, the communication method comprising thesteps of: determining whether or not the receiving apparatus can receivea sub-signal on the basis of the capability information; adding thesub-signal to pixel data of a main image composed of pixel data whosenumber of bits is smaller than that of transmission pixel data that ispixel data to be transmitted by the transmission means when thereceiving apparatus can receive the sub-signal, thereby constructing thetransmission pixel data; and allowing sub-signal information indicatingwhether or not the sub-signal is contained in the transmission pixeldata transmitted in the valid image period immediately after thevertical blanking period, to be transmitted in the vertical blankingperiod.
 5. A program to be executed by a computer for controlling atransmission apparatus for transmitting, after capability informationindicating the capability of a receiving apparatus is received, pixeldata of an uncompressed image for one screen in one direction to thereceiving apparatus by using a differential signal through a pluralityof channels for transmitting data of a fixed number of bits per clock ofa pixel clock in a valid image period that is a period in which ahorizontal blanking period and a vertical blanking period are excludedfrom a period from one vertical synchronization signal to the nextvertical synchronization signal, the transmission apparatus includingtransmission means for transmitting pixel data to which a number ofbits, which is greater than the fixed number of bits, is assigned in onedirection to the receiving apparatus by using a differential signalthrough the plurality of channels by adjusting the frequency of thepixel clock, the program enabling a computer to perform communicationprocessing comprising the steps of: determining whether or not thereceiving apparatus can receive a sub-signal on the basis of thecapability information; adding the sub-signal to pixel data of a mainimage composed of pixel data whose number of bits is smaller than thatof transmission pixel data that is pixel data to be transmitted by thetransmission means when the receiving apparatus can receive thesub-signal, thereby constructing the transmission pixel data; andallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period.
 6. A receiving apparatusfor receiving pixel data transmitted using a differential signal througha plurality of channels from a transmission apparatus after capabilityinformation is transmitted to the transmission apparatus for, after thecapability information indicating the capability of the receivingapparatus is received, transmitting pixel data of an uncompressed imagefor one screen in one direction to the receiving apparatus by using adifferential signal through a plurality of channels for transmittingdata of a fixed number of bits per clock of a pixel clock in a validimage period that is a period in which a horizontal blanking period anda vertical blanking period are excluded from a period from one verticalsynchronization signal to the next vertical synchronization signal, thetransmission apparatus including transmission means for transmittingpixel data to which a number of bits, which is greater than the fixednumber of bits, is assigned, in one direction to the receiving apparatusby using a differential signal through the plurality of channels byadjusting the frequency of the pixel clock; sub-signal receptioncapability/incapability determination means for determining whether ornot the receiving apparatus can receive a sub-signal on the basis of thecapability information; sub-signal addition means for adding thesub-signal to pixel data of a main image composed of pixel data whosenumber of bits is smaller than that of transmission pixel data that ispixel data to be transmitted by the transmission means when thereceiving apparatus can receive the sub-signal, thereby constructing thetransmission pixel data; and information transmission control means forallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period, the receiving apparatuscomprising: receiving means for receiving the transmission pixel datatransmitted using a differential signal through the plurality ofchannels; sub-signal presence/absence determination means fordetermining whether or not the sub-signal is contained in thetransmission pixel data transmitted in a valid image period immediatelyafter the vertical blanking period on the basis of the sub-signalinformation transmitted in the vertical blanking period; and separationmeans for separating the sub-signal from the transmission pixel datawhen the sub-signal is contained in the transmission pixel data.
 7. Acommunication method for use with a receiving apparatus for receivingpixel data transmitted using a differential signal through a pluralityof channels from a transmission apparatus after capability informationis transmitted to the transmission apparatus for, after the capabilityinformation indicating the capability of the receiving apparatus isreceived, transmitting pixel data of an uncompressed image for onescreen in one direction to the receiving apparatus by using adifferential signal through a plurality of channels for transmittingdata of a fixed number of bits per clock of a pixel clock in a validimage period that is a period in which a horizontal blanking period anda vertical blanking period are excluded from a period from one verticalsynchronization signal to the next vertical synchronization signal, thetransmission apparatus including transmission means for transmittingpixel data to which a number of bits, which is greater than the fixednumber of bits, is assigned, in one direction to the receiving apparatusby using a differential signal through the plurality of channels byadjusting the frequency of the pixel clock; sub-signal receptioncapability/incapability determination means for determining whether ornot the receiving apparatus can receive a sub-signal on the basis of thecapability information; sub-signal addition means for adding thesub-signal to pixel data of a main image composed of pixel data whosenumber of bits is smaller than that of transmission pixel data that ispixel data to be transmitted by the transmission means when thereceiving apparatus can receive the sub-signal, thereby constructing thetransmission pixel data; and information transmission control means forallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period, the receiving apparatusincluding receiving means for receiving the transmission pixel datatransmitted using a differential signal through the plurality ofchannels, the communication method comprising the steps of: determiningwhether or not the sub-signal is contained in the transmission pixeldata transmitted in a valid image period immediately after the verticalblanking period on the basis of the sub-signal information transmittedin the vertical blanking period; and separating the sub-signal from thetransmission pixel data when the sub-signal is contained in thetransmission pixel data.
 8. A program to be executed by a computer forcontrolling a receiving apparatus for receiving pixel data transmittedusing a differential signal through a plurality of channels from atransmission apparatus after capability information is transmitted tothe transmission apparatus for, after capability information indicatingthe capability of the receiving apparatus is received, transmittingpixel data of an uncompressed image for one screen in one direction tothe receiving apparatus by using a differential signal through aplurality of channels for transmitting data of a fixed number of bitsper clock of a pixel clock in a valid image period that is a period inwhich a horizontal blanking period and a vertical blanking period areexcluded from a period from one vertical synchronization signal to thenext vertical synchronization signal, the transmission apparatusincluding transmission means for transmitting pixel data to which anumber of bits, which is greater than the fixed number of bits, isassigned, in one direction to the receiving apparatus by using adifferential signal through the plurality of channels by adjusting thefrequency of the pixel clock; sub-signal receptioncapability/incapability determination means for determining whether ornot the receiving apparatus can receive a sub-signal on the basis of thecapability information; sub-signal addition means for adding thesub-signal to pixel data of a main image composed of pixel data whosenumber of bits is smaller than that of transmission pixel data that ispixel data to be transmitted by the transmission means when thereceiving apparatus can receive the sub-signal, thereby constructing thetransmission pixel data; and information transmission control means forallowing sub-signal information indicating whether or not the sub-signalis contained in the transmission pixel data transmitted in the validimage period immediately after the vertical blanking period, to betransmitted in the vertical blanking period, the receiving apparatusincluding receiving means for receiving the transmission pixel datatransmitted using a differential signal through the plurality ofchannels, the program enabling a computer to perform communicationprocessing comprising the steps of: determining whether or not thesub-signal is contained in the transmission pixel data transmitted in avalid image period immediately after the vertical blanking period on thebasis of the sub-signal information transmitted in the vertical blankingperiod; and separating the sub-signal from the transmission pixel datawhen the sub-signal is contained in the transmission pixel data.
 9. Atransmission method for transmitting video data by using a transmissionscheme for transmitting video data in units of 8 bits from a source-sideapparatus to a sink-side apparatus by using individual transmissionlines for each piece of color data or for each of luminance andcolor-difference signals in synchronization with a pixel clock, thetransmission method comprising the steps of: setting video data for onepixel to be transmitted from the source-side apparatus to the sink-sideapparatus to a predetermined number of bits that is not an integralmultiple of 8 bits; transmitting the video data of the predeterminednumber of bits that is not an integral multiple of 8 bits at a timingsynchronized with the pixel clock; and arranging data differing from thevideo data of the predetermined number of bits in a marginaltransmission period that occurs in an amount corresponding to the numberof bits of the difference between the number of bits transmitted in theperiod of the pixel clock assigned to the transmission of one pixel andthe predetermined number of bits, and transmitting the data from thesource-side apparatus to the sink-side apparatus.
 10. The transmissionmethod according to claim 9, wherein the source-side apparatus encryptsdata to be sent to each transmission line in units of 8 bits, and thesink-side apparatus decrypts the data received via each transmissionline in units of 8 bits.
 11. The transmission method according to claim9, wherein the other data is sub-video data differing from the videodata.
 12. The transmission method according to claim 9, wherein theother data is audio data.
 13. The transmission method according to claim9, wherein the other data is auxiliary information.
 14. A transmissionsystem for transmitting video data by using a transmission scheme fortransmitting video data in units of 8 bits from a source-side apparatusto a sink-side apparatus by using individual transmission lines for eachpiece of color data or for each of luminance and color-differencesignals in synchronization with a pixel clock, wherein the source-sideapparatus includes a video data generator configured to generate videodata of a predetermined number of bits such that video data of one pixelis not an integral multiple of 8 bits; a transmission data arrangementsection configured to arrange video data of the predetermined number ofbits, the video data being generated by the video data generator, in avideo data transmission period in units of 8 bits of the transmissionscheme, and configured to arrange data differing from the video data ofthe predetermined number of bits in a marginal transmission period thatoccurs in an amount corresponding to the number of bits of thedifference between the number of bits transmitted in the period of thepixel clock assigned to the transmission of one pixel and thepredetermined number of bits, thereby constructing data in units of 8bits; and a sending section configured to send the data in units of 8bits arranged by the transmission data arrangement section to thetransmission line in synchronization with the pixel clock, and, whereinthe sink-side apparatus includes a receiver configured to detect thedata received from the transmission line in units of 8 bits insynchronization with the pixel clock; and a data separator configured toseparate the video data of the predetermined number of bits and theother data transmitted in the marginal transmission period from the datain units of 8 bits received by the receiver.
 15. The transmission systemaccording to claim 9, wherein the source-side apparatus comprises anencryptor configured to encrypt the transmission data arranged by thetransmission data arrangement section in units of 8 bits, and thetransmission data encrypted by the encryptor is sent from the sendingsection, and the sink-side apparatus comprises a decryptor configured todecrypt the transmission data received by the receiver in units of 8bits, and the transmission data decrypted by the decryptor is separatedby the data separator.
 16. A transmission method for transmitting videodata by using a transmission scheme for transmitting video data in unitsof 8 bits by using individual transmission lines for each piece of colordata or for each of luminance and color-difference signals insynchronization with a pixel clock, the transmission method comprisingthe steps of: setting video data of one pixel to be transmitted to apredetermined number of bits that is not an integral multiple of 8 bits;transmitting the video data of the predetermined number of bits that isnot an integral multiple of 8 bits at a timing synchronized with thepixel clock; and arranging data differing from the video data of thepredetermined number of bits in a marginal transmission period thatoccurs in an amount corresponding to the number of bits of thedifference between the number of bits transmitted in the period of thepixel clock assigned to the transmission of one pixel and thepredetermined number of bits, and transmitting the data from asource-side apparatus to a sink-side apparatus.
 17. A transmissionapparatus for transmitting video data by using a transmission scheme fortransmitting video data in units of 8 bits by using individualtransmission lines for each piece of color data or for each of luminanceand color-difference signals in synchronization with a pixel clock, thetransmission apparatus comprising: a video data generator configured togenerate video data of a predetermined number of bits such that videodata of one pixel is not an integral multiple of 8 bits; a transmissiondata arrangement section configured to arrange the video data of thepredetermined number of bits, the video data being generated by thevideo data generator, in a video data transmission period in units of 8bits of the transmission scheme, and configured to arrange datadiffering from the video data of the predetermined number of bits in amarginal transmission period that occurs in an amount corresponding tothe number of bits of the difference between the number of bitstransmitted in the period of the pixel clock assigned to thetransmission of one pixel and the predetermined number of bits, therebyconstructing data in units of 8 bits; and a sending section configuredto send the data in units of 8 bits arranged by the transmission dataarrangement section to the transmission lines in synchronization withthe pixel clock.
 18. The transmission apparatus according to claim 17,further comprising an encryptor configured to encrypt transmission dataarranged by the transmission data arrangement section in units of 8bits, wherein the transmission data encrypted by the encryptor is sentfrom the sending section.
 19. A receiving method for receiving videodata by using a transmission scheme for transmitting video data by usingindividual transmission lines for each piece of color data or for eachof luminance and color-difference signals in synchronization with apixel clock, the receiving method comprising the steps of: setting thevideo data of one pixel to be received to a predetermined number of bitsthat is not an integral multiple of 8 bits and receiving the video dataof the predetermined number of bits that is not an integral multiple of8 bits in synchronization with a timing synchronized with the pixelclock; and separating data differing from the video data of thepredetermined number of bits, the video data being arranged in amarginal transmission period that occurs in an amount corresponding tothe number of bits of the difference between the number of bitstransmitted in the period of a pixel clock assigned to the transmissionof one pixel and the predetermined number of bits.
 20. A receivingapparatus for receiving video data by using a transmission scheme fortransmitting the video data by using individual transmission lines foreach piece of color data or for each of luminance and color-differencesignals in synchronization with a pixel clock, the receiving apparatuscomprising: a receiver configured to detect the data received from thetransmission lines in units of 8 bits in synchronization with the pixelclock; and a data separator configured to separate, from the data inunits of 8 bits received by the receiver, the video data of thepredetermined number of bits, which is not an integral multiple of 8bits, and the other data transmitted in the marginal transmission periodthat occurs in an amount corresponding to the number of bits of thedifference between the number of bits transmitted in the period of apixel clock assigned to the transmission of one pixel and thepredetermined number of bits.
 21. The receiving apparatus according toclaim 20, further comprising a decryptor configured to decrypt thetransmission data received by the receiver in units of 8 bits, whereinthe transmission data decrypted by the decryptor is separated by thedata separator.